split TOAST support out of postgres.h
Most backend code doesn't actually need the variable-length data types
support (TOAST support) in postgres.h. So I figured we could try to put
it into a separate header file. That makes postgres.h more manageable,
and it avoids including a bunch of complicated unused stuff everywhere.
I picked "varatt.h" as the name. Then we could either
1) Include varatt.h in postgres.h, similar to elog.h and palloc.h. That
way we clean up the files a bit but don't change any external interfaces.
2) Just let everyone who needs it include the new file.
3) Compromise: You can avoid most "damage" by having fmgr.h include
varatt.h. That satisfies most data types and extension code. That way,
there are only a few places that need an explicit include of varatt.h.
I went with the last option in my patch.
Thoughts?
Attachments:
0001-New-header-varatt.h-split-off-from-postgres.h.patchtext/plain; charset=UTF-8; name=0001-New-header-varatt.h-split-off-from-postgres.h.patchDownload
From fba83e468a8ea881f0bdd6cf49aef13b71f67273 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Wed, 28 Dec 2022 13:30:05 +0100
Subject: [PATCH] New header varatt.h split off from postgres.h
This new header contains all the variable-length data types support
(TOAST support) from postgres.h, which isn't needed by large parts of
the backend code.
This new header file is included by fmgr.h, so most data type and
extension code won't need any changes.
---
contrib/cube/cubedata.h | 2 +
contrib/pg_trgm/trgm_regexp.c | 1 +
src/backend/access/table/toast_helper.c | 1 +
src/backend/libpq/pqformat.c | 1 +
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 2 +
src/include/meson.build | 1 +
src/include/postgres.h | 358 +--------------
src/include/utils/expandeddatum.h | 2 +
src/include/{postgres.h => varatt.h} | 579 +-----------------------
10 files changed, 21 insertions(+), 927 deletions(-)
copy src/include/{postgres.h => varatt.h} (53%)
diff --git a/contrib/cube/cubedata.h b/contrib/cube/cubedata.h
index 96fa41a04e..d8df478059 100644
--- a/contrib/cube/cubedata.h
+++ b/contrib/cube/cubedata.h
@@ -1,5 +1,7 @@
/* contrib/cube/cubedata.h */
+#include "fmgr.h"
+
/*
* This limit is pretty arbitrary, but don't make it so large that you
* risk overflow in sizing calculations.
diff --git a/contrib/pg_trgm/trgm_regexp.c b/contrib/pg_trgm/trgm_regexp.c
index 58d32ba946..08eecd269b 100644
--- a/contrib/pg_trgm/trgm_regexp.c
+++ b/contrib/pg_trgm/trgm_regexp.c
@@ -196,6 +196,7 @@
#include "tsearch/ts_locale.h"
#include "utils/hsearch.h"
#include "utils/memutils.h"
+#include "varatt.h"
/*
* Uncomment (or use -DTRGM_REGEXP_DEBUG) to print debug info,
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 74ba2189f0..a4889d7642 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -19,6 +19,7 @@
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
+#include "varatt.h"
/*
diff --git a/src/backend/libpq/pqformat.c b/src/backend/libpq/pqformat.c
index 9c24df3360..65a913e893 100644
--- a/src/backend/libpq/pqformat.c
+++ b/src/backend/libpq/pqformat.c
@@ -77,6 +77,7 @@
#include "libpq/pqformat.h"
#include "mb/pg_wchar.h"
#include "port/pg_bswap.h"
+#include "varatt.h"
/* --------------------------------
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index c1af814e8d..c7b505e33f 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -19,6 +19,7 @@
#include "access/tupdesc.h"
#include "access/tupmacs.h"
#include "storage/bufpage.h"
+#include "varatt.h"
/*
* MaxTupleAttributeNumber limits the number of (user) columns in a tuple.
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index 972afe3aff..0ce0090360 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -18,6 +18,8 @@
#ifndef FMGR_H
#define FMGR_H
+#include "varatt.h"
+
/* We don't want to include primnodes.h here, so make some stub references */
typedef struct Node *fmNodePtr;
typedef struct Aggref *fmAggrefPtr;
diff --git a/src/include/meson.build b/src/include/meson.build
index b4820049c8..63b444bc99 100644
--- a/src/include/meson.build
+++ b/src/include/meson.build
@@ -113,6 +113,7 @@ install_headers(
'postgres.h',
'postgres_ext.h',
'postgres_fe.h',
+ 'varatt.h',
'windowapi.h',
pg_config_ext,
pg_config_os,
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 54730dfb38..a915f2b97f 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -23,9 +23,8 @@
*
* section description
* ------- ------------------------------------------------
- * 1) variable-length datatypes (TOAST support)
- * 2) Datum type + support functions
- * 3) miscellaneous
+ * 1) Datum type + support functions
+ * 2) miscellaneous
*
* NOTES
*
@@ -36,8 +35,7 @@
* postgres_fe.h. We do that since those type definitions are needed by
* frontend modules that want to deal with binary data transmission to or
* from the backend. Type definitions in this file should be for
- * representations that never escape the backend, such as Datum or
- * TOASTed varlena objects.
+ * representations that never escape the backend, such as Datum.
*
*----------------------------------------------------------------
*/
@@ -49,353 +47,7 @@
#include "utils/palloc.h"
/* ----------------------------------------------------------------
- * Section 1: variable-length datatypes (TOAST support)
- * ----------------------------------------------------------------
- */
-
-/*
- * struct varatt_external is a traditional "TOAST pointer", that is, the
- * information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if the external size stored in
- * va_extinfo is less than va_rawsize - VARHDRSZ.
- *
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
- *
- * Note that this information is stored unaligned within actual tuples, so
- * you need to memcpy from the tuple into a local struct variable before
- * you can look at these fields! (The reason we use memcmp is to avoid
- * having to do that just to detect equality of two TOAST pointers...)
- */
-typedef struct varatt_external
-{
- int32 va_rawsize; /* Original data size (includes header) */
- uint32 va_extinfo; /* External saved size (without header) and
- * compression method */
- Oid va_valueid; /* Unique ID of value within TOAST table */
- Oid va_toastrelid; /* RelID of TOAST table containing it */
-} varatt_external;
-
-/*
- * These macros define the "saved size" portion of va_extinfo. Its remaining
- * two high-order bits identify the compression method.
- */
-#define VARLENA_EXTSIZE_BITS 30
-#define VARLENA_EXTSIZE_MASK ((1U << VARLENA_EXTSIZE_BITS) - 1)
-
-/*
- * struct varatt_indirect is a "TOAST pointer" representing an out-of-line
- * Datum that's stored in memory, not in an external toast relation.
- * The creator of such a Datum is entirely responsible that the referenced
- * storage survives for as long as referencing pointer Datums can exist.
- *
- * Note that just as for struct varatt_external, this struct is stored
- * unaligned within any containing tuple.
- */
-typedef struct varatt_indirect
-{
- struct varlena *pointer; /* Pointer to in-memory varlena */
-} varatt_indirect;
-
-/*
- * struct varatt_expanded is a "TOAST pointer" representing an out-of-line
- * Datum that is stored in memory, in some type-specific, not necessarily
- * physically contiguous format that is convenient for computation not
- * storage. APIs for this, in particular the definition of struct
- * ExpandedObjectHeader, are in src/include/utils/expandeddatum.h.
- *
- * Note that just as for struct varatt_external, this struct is stored
- * unaligned within any containing tuple.
- */
-typedef struct ExpandedObjectHeader ExpandedObjectHeader;
-
-typedef struct varatt_expanded
-{
- ExpandedObjectHeader *eohptr;
-} varatt_expanded;
-
-/*
- * Type tag for the various sorts of "TOAST pointer" datums. The peculiar
- * value for VARTAG_ONDISK comes from a requirement for on-disk compatibility
- * with a previous notion that the tag field was the pointer datum's length.
- */
-typedef enum vartag_external
-{
- VARTAG_INDIRECT = 1,
- VARTAG_EXPANDED_RO = 2,
- VARTAG_EXPANDED_RW = 3,
- VARTAG_ONDISK = 18
-} vartag_external;
-
-/* this test relies on the specific tag values above */
-#define VARTAG_IS_EXPANDED(tag) \
- (((tag) & ~1) == VARTAG_EXPANDED_RO)
-
-#define VARTAG_SIZE(tag) \
- ((tag) == VARTAG_INDIRECT ? sizeof(varatt_indirect) : \
- VARTAG_IS_EXPANDED(tag) ? sizeof(varatt_expanded) : \
- (tag) == VARTAG_ONDISK ? sizeof(varatt_external) : \
- (AssertMacro(false), 0))
-
-/*
- * These structs describe the header of a varlena object that may have been
- * TOASTed. Generally, don't reference these structs directly, but use the
- * macros below.
- *
- * We use separate structs for the aligned and unaligned cases because the
- * compiler might otherwise think it could generate code that assumes
- * alignment while touching fields of a 1-byte-header varlena.
- */
-typedef union
-{
- struct /* Normal varlena (4-byte length) */
- {
- uint32 va_header;
- char va_data[FLEXIBLE_ARRAY_MEMBER];
- } va_4byte;
- struct /* Compressed-in-line format */
- {
- uint32 va_header;
- uint32 va_tcinfo; /* Original data size (excludes header) and
- * compression method; see va_extinfo */
- char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
- } va_compressed;
-} varattrib_4b;
-
-typedef struct
-{
- uint8 va_header;
- char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Data begins here */
-} varattrib_1b;
-
-/* TOAST pointers are a subset of varattrib_1b with an identifying tag byte */
-typedef struct
-{
- uint8 va_header; /* Always 0x80 or 0x01 */
- uint8 va_tag; /* Type of datum */
- char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Type-specific data */
-} varattrib_1b_e;
-
-/*
- * Bit layouts for varlena headers on big-endian machines:
- *
- * 00xxxxxx 4-byte length word, aligned, uncompressed data (up to 1G)
- * 01xxxxxx 4-byte length word, aligned, *compressed* data (up to 1G)
- * 10000000 1-byte length word, unaligned, TOAST pointer
- * 1xxxxxxx 1-byte length word, unaligned, uncompressed data (up to 126b)
- *
- * Bit layouts for varlena headers on little-endian machines:
- *
- * xxxxxx00 4-byte length word, aligned, uncompressed data (up to 1G)
- * xxxxxx10 4-byte length word, aligned, *compressed* data (up to 1G)
- * 00000001 1-byte length word, unaligned, TOAST pointer
- * xxxxxxx1 1-byte length word, unaligned, uncompressed data (up to 126b)
- *
- * The "xxx" bits are the length field (which includes itself in all cases).
- * In the big-endian case we mask to extract the length, in the little-endian
- * case we shift. Note that in both cases the flag bits are in the physically
- * first byte. Also, it is not possible for a 1-byte length word to be zero;
- * this lets us disambiguate alignment padding bytes from the start of an
- * unaligned datum. (We now *require* pad bytes to be filled with zero!)
- *
- * In TOAST pointers the va_tag field (see varattrib_1b_e) is used to discern
- * the specific type and length of the pointer datum.
- */
-
-/*
- * Endian-dependent macros. These are considered internal --- use the
- * external macros below instead of using these directly.
- *
- * Note: IS_1B is true for external toast records but VARSIZE_1B will return 0
- * for such records. Hence you should usually check for IS_EXTERNAL before
- * checking for IS_1B.
- */
-
-#ifdef WORDS_BIGENDIAN
-
-#define VARATT_IS_4B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x80) == 0x00)
-#define VARATT_IS_4B_U(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0xC0) == 0x00)
-#define VARATT_IS_4B_C(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0xC0) == 0x40)
-#define VARATT_IS_1B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x80) == 0x80)
-#define VARATT_IS_1B_E(PTR) \
- ((((varattrib_1b *) (PTR))->va_header) == 0x80)
-#define VARATT_NOT_PAD_BYTE(PTR) \
- (*((uint8 *) (PTR)) != 0)
-
-/* VARSIZE_4B() should only be used on known-aligned data */
-#define VARSIZE_4B(PTR) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header & 0x3FFFFFFF)
-#define VARSIZE_1B(PTR) \
- (((varattrib_1b *) (PTR))->va_header & 0x7F)
-#define VARTAG_1B_E(PTR) \
- (((varattrib_1b_e *) (PTR))->va_tag)
-
-#define SET_VARSIZE_4B(PTR,len) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header = (len) & 0x3FFFFFFF)
-#define SET_VARSIZE_4B_C(PTR,len) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header = ((len) & 0x3FFFFFFF) | 0x40000000)
-#define SET_VARSIZE_1B(PTR,len) \
- (((varattrib_1b *) (PTR))->va_header = (len) | 0x80)
-#define SET_VARTAG_1B_E(PTR,tag) \
- (((varattrib_1b_e *) (PTR))->va_header = 0x80, \
- ((varattrib_1b_e *) (PTR))->va_tag = (tag))
-
-#else /* !WORDS_BIGENDIAN */
-
-#define VARATT_IS_4B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x01) == 0x00)
-#define VARATT_IS_4B_U(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x03) == 0x00)
-#define VARATT_IS_4B_C(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x03) == 0x02)
-#define VARATT_IS_1B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x01) == 0x01)
-#define VARATT_IS_1B_E(PTR) \
- ((((varattrib_1b *) (PTR))->va_header) == 0x01)
-#define VARATT_NOT_PAD_BYTE(PTR) \
- (*((uint8 *) (PTR)) != 0)
-
-/* VARSIZE_4B() should only be used on known-aligned data */
-#define VARSIZE_4B(PTR) \
- ((((varattrib_4b *) (PTR))->va_4byte.va_header >> 2) & 0x3FFFFFFF)
-#define VARSIZE_1B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header >> 1) & 0x7F)
-#define VARTAG_1B_E(PTR) \
- (((varattrib_1b_e *) (PTR))->va_tag)
-
-#define SET_VARSIZE_4B(PTR,len) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header = (((uint32) (len)) << 2))
-#define SET_VARSIZE_4B_C(PTR,len) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header = (((uint32) (len)) << 2) | 0x02)
-#define SET_VARSIZE_1B(PTR,len) \
- (((varattrib_1b *) (PTR))->va_header = (((uint8) (len)) << 1) | 0x01)
-#define SET_VARTAG_1B_E(PTR,tag) \
- (((varattrib_1b_e *) (PTR))->va_header = 0x01, \
- ((varattrib_1b_e *) (PTR))->va_tag = (tag))
-
-#endif /* WORDS_BIGENDIAN */
-
-#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
-#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
-#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
-#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
-
-/*
- * Externally visible TOAST macros begin here.
- */
-
-#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
-#define VARHDRSZ_COMPRESSED offsetof(varattrib_4b, va_compressed.va_data)
-#define VARHDRSZ_SHORT offsetof(varattrib_1b, va_data)
-
-#define VARATT_SHORT_MAX 0x7F
-#define VARATT_CAN_MAKE_SHORT(PTR) \
- (VARATT_IS_4B_U(PTR) && \
- (VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT) <= VARATT_SHORT_MAX)
-#define VARATT_CONVERTED_SHORT_SIZE(PTR) \
- (VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
-
-/*
- * In consumers oblivious to data alignment, call PG_DETOAST_DATUM_PACKED(),
- * VARDATA_ANY(), VARSIZE_ANY() and VARSIZE_ANY_EXHDR(). Elsewhere, call
- * PG_DETOAST_DATUM(), VARDATA() and VARSIZE(). Directly fetching an int16,
- * int32 or wider field in the struct representing the datum layout requires
- * aligned data. memcpy() is alignment-oblivious, as are most operations on
- * datatypes, such as text, whose layout struct contains only char fields.
- *
- * Code assembling a new datum should call VARDATA() and SET_VARSIZE().
- * (Datums begin life untoasted.)
- *
- * Other macros here should usually be used only by tuple assembly/disassembly
- * code and code that specifically wants to work with still-toasted Datums.
- */
-#define VARDATA(PTR) VARDATA_4B(PTR)
-#define VARSIZE(PTR) VARSIZE_4B(PTR)
-
-#define VARSIZE_SHORT(PTR) VARSIZE_1B(PTR)
-#define VARDATA_SHORT(PTR) VARDATA_1B(PTR)
-
-#define VARTAG_EXTERNAL(PTR) VARTAG_1B_E(PTR)
-#define VARSIZE_EXTERNAL(PTR) (VARHDRSZ_EXTERNAL + VARTAG_SIZE(VARTAG_EXTERNAL(PTR)))
-#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
-
-#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
-#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
-#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
-#define VARATT_IS_EXTERNAL_INDIRECT(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_INDIRECT)
-#define VARATT_IS_EXTERNAL_EXPANDED_RO(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_EXPANDED_RO)
-#define VARATT_IS_EXTERNAL_EXPANDED_RW(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_EXPANDED_RW)
-#define VARATT_IS_EXTERNAL_EXPANDED(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
-#define VARATT_IS_EXTERNAL_NON_EXPANDED(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && !VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
-#define VARATT_IS_SHORT(PTR) VARATT_IS_1B(PTR)
-#define VARATT_IS_EXTENDED(PTR) (!VARATT_IS_4B_U(PTR))
-
-#define SET_VARSIZE(PTR, len) SET_VARSIZE_4B(PTR, len)
-#define SET_VARSIZE_SHORT(PTR, len) SET_VARSIZE_1B(PTR, len)
-#define SET_VARSIZE_COMPRESSED(PTR, len) SET_VARSIZE_4B_C(PTR, len)
-
-#define SET_VARTAG_EXTERNAL(PTR, tag) SET_VARTAG_1B_E(PTR, tag)
-
-#define VARSIZE_ANY(PTR) \
- (VARATT_IS_1B_E(PTR) ? VARSIZE_EXTERNAL(PTR) : \
- (VARATT_IS_1B(PTR) ? VARSIZE_1B(PTR) : \
- VARSIZE_4B(PTR)))
-
-/* Size of a varlena data, excluding header */
-#define VARSIZE_ANY_EXHDR(PTR) \
- (VARATT_IS_1B_E(PTR) ? VARSIZE_EXTERNAL(PTR)-VARHDRSZ_EXTERNAL : \
- (VARATT_IS_1B(PTR) ? VARSIZE_1B(PTR)-VARHDRSZ_SHORT : \
- VARSIZE_4B(PTR)-VARHDRSZ))
-
-/* caution: this will not work on an external or compressed-in-line Datum */
-/* caution: this will return a possibly unaligned pointer */
-#define VARDATA_ANY(PTR) \
- (VARATT_IS_1B(PTR) ? VARDATA_1B(PTR) : VARDATA_4B(PTR))
-
-/* Decompressed size and compression method of a compressed-in-line Datum */
-#define VARDATA_COMPRESSED_GET_EXTSIZE(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_EXTSIZE_MASK)
-#define VARDATA_COMPRESSED_GET_COMPRESS_METHOD(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS)
-
-/* Same for external Datums; but note argument is a struct varatt_external */
-#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
- ((toast_pointer).va_extinfo & VARLENA_EXTSIZE_MASK)
-#define VARATT_EXTERNAL_GET_COMPRESS_METHOD(toast_pointer) \
- ((toast_pointer).va_extinfo >> VARLENA_EXTSIZE_BITS)
-
-#define VARATT_EXTERNAL_SET_SIZE_AND_COMPRESS_METHOD(toast_pointer, len, cm) \
- do { \
- Assert((cm) == TOAST_PGLZ_COMPRESSION_ID || \
- (cm) == TOAST_LZ4_COMPRESSION_ID); \
- ((toast_pointer).va_extinfo = \
- (len) | ((uint32) (cm) << VARLENA_EXTSIZE_BITS)); \
- } while (0)
-
-/*
- * Testing whether an externally-stored value is compressed now requires
- * comparing size stored in va_extinfo (the actual length of the external data)
- * to rawsize (the original uncompressed datum's size). The latter includes
- * VARHDRSZ overhead, the former doesn't. We never use compression unless it
- * actually saves space, so we expect either equality or less-than.
- */
-#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < \
- (toast_pointer).va_rawsize - VARHDRSZ)
-
-
-/* ----------------------------------------------------------------
- * Section 2: Datum type + support functions
+ * Section 1: Datum type + support functions
* ----------------------------------------------------------------
*/
@@ -907,7 +559,7 @@ extern Datum Float8GetDatum(float8 X);
/* ----------------------------------------------------------------
- * Section 3: miscellaneous
+ * Section 2: miscellaneous
* ----------------------------------------------------------------
*/
diff --git a/src/include/utils/expandeddatum.h b/src/include/utils/expandeddatum.h
index c4818eb054..139f49d940 100644
--- a/src/include/utils/expandeddatum.h
+++ b/src/include/utils/expandeddatum.h
@@ -44,6 +44,8 @@
#ifndef EXPANDEDDATUM_H
#define EXPANDEDDATUM_H
+#include "varatt.h"
+
/* Size of an EXTERNAL datum that contains a pointer to an expanded object */
#define EXPANDED_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_expanded))
diff --git a/src/include/postgres.h b/src/include/varatt.h
similarity index 53%
copy from src/include/postgres.h
copy to src/include/varatt.h
index 54730dfb38..37a139145e 100644
--- a/src/include/postgres.h
+++ b/src/include/varatt.h
@@ -1,57 +1,19 @@
/*-------------------------------------------------------------------------
*
- * postgres.h
- * Primary include file for PostgreSQL server .c files
- *
- * This should be the first file included by PostgreSQL backend modules.
- * Client-side code should include postgres_fe.h instead.
+ * varatt.h
+ * variable-length datatypes (TOAST support)
*
*
* Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
* Portions Copyright (c) 1995, Regents of the University of California
*
- * src/include/postgres.h
+ * src/include/varatt.h
*
*-------------------------------------------------------------------------
*/
-/*
- *----------------------------------------------------------------
- * TABLE OF CONTENTS
- *
- * When adding stuff to this file, please try to put stuff
- * into the relevant section, or add new sections as appropriate.
- *
- * section description
- * ------- ------------------------------------------------
- * 1) variable-length datatypes (TOAST support)
- * 2) Datum type + support functions
- * 3) miscellaneous
- *
- * NOTES
- *
- * In general, this file should contain declarations that are widely needed
- * in the backend environment, but are of no interest outside the backend.
- *
- * Simple type definitions live in c.h, where they are shared with
- * postgres_fe.h. We do that since those type definitions are needed by
- * frontend modules that want to deal with binary data transmission to or
- * from the backend. Type definitions in this file should be for
- * representations that never escape the backend, such as Datum or
- * TOASTed varlena objects.
- *
- *----------------------------------------------------------------
- */
-#ifndef POSTGRES_H
-#define POSTGRES_H
-#include "c.h"
-#include "utils/elog.h"
-#include "utils/palloc.h"
-
-/* ----------------------------------------------------------------
- * Section 1: variable-length datatypes (TOAST support)
- * ----------------------------------------------------------------
- */
+#ifndef VARATT_H
+#define VARATT_H
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
@@ -393,535 +355,4 @@ typedef struct
(VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < \
(toast_pointer).va_rawsize - VARHDRSZ)
-
-/* ----------------------------------------------------------------
- * Section 2: Datum type + support functions
- * ----------------------------------------------------------------
- */
-
-/*
- * A Datum contains either a value of a pass-by-value type or a pointer to a
- * value of a pass-by-reference type. Therefore, we require:
- *
- * sizeof(Datum) == sizeof(void *) == 4 or 8
- *
- * The functions below and the analogous functions for other types should be used to
- * convert between a Datum and the appropriate C type.
- */
-
-typedef uintptr_t Datum;
-
-/*
- * A NullableDatum is used in places where both a Datum and its nullness needs
- * to be stored. This can be more efficient than storing datums and nullness
- * in separate arrays, due to better spatial locality, even if more space may
- * be wasted due to padding.
- */
-typedef struct NullableDatum
-{
-#define FIELDNO_NULLABLE_DATUM_DATUM 0
- Datum value;
-#define FIELDNO_NULLABLE_DATUM_ISNULL 1
- bool isnull;
- /* due to alignment padding this could be used for flags for free */
-} NullableDatum;
-
-#define SIZEOF_DATUM SIZEOF_VOID_P
-
-/*
- * DatumGetBool
- * Returns boolean value of a datum.
- *
- * Note: any nonzero value will be considered true.
- */
-static inline bool
-DatumGetBool(Datum X)
-{
- return (X != 0);
-}
-
-/*
- * BoolGetDatum
- * Returns datum representation for a boolean.
- *
- * Note: any nonzero value will be considered true.
- */
-static inline Datum
-BoolGetDatum(bool X)
-{
- return (Datum) (X ? 1 : 0);
-}
-
-/*
- * DatumGetChar
- * Returns character value of a datum.
- */
-static inline char
-DatumGetChar(Datum X)
-{
- return (char) X;
-}
-
-/*
- * CharGetDatum
- * Returns datum representation for a character.
- */
-static inline Datum
-CharGetDatum(char X)
-{
- return (Datum) X;
-}
-
-/*
- * Int8GetDatum
- * Returns datum representation for an 8-bit integer.
- */
-static inline Datum
-Int8GetDatum(int8 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetUInt8
- * Returns 8-bit unsigned integer value of a datum.
- */
-static inline uint8
-DatumGetUInt8(Datum X)
-{
- return (uint8) X;
-}
-
-/*
- * UInt8GetDatum
- * Returns datum representation for an 8-bit unsigned integer.
- */
-static inline Datum
-UInt8GetDatum(uint8 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetInt16
- * Returns 16-bit integer value of a datum.
- */
-static inline int16
-DatumGetInt16(Datum X)
-{
- return (int16) X;
-}
-
-/*
- * Int16GetDatum
- * Returns datum representation for a 16-bit integer.
- */
-static inline Datum
-Int16GetDatum(int16 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetUInt16
- * Returns 16-bit unsigned integer value of a datum.
- */
-static inline uint16
-DatumGetUInt16(Datum X)
-{
- return (uint16) X;
-}
-
-/*
- * UInt16GetDatum
- * Returns datum representation for a 16-bit unsigned integer.
- */
-static inline Datum
-UInt16GetDatum(uint16 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetInt32
- * Returns 32-bit integer value of a datum.
- */
-static inline int32
-DatumGetInt32(Datum X)
-{
- return (int32) X;
-}
-
-/*
- * Int32GetDatum
- * Returns datum representation for a 32-bit integer.
- */
-static inline Datum
-Int32GetDatum(int32 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetUInt32
- * Returns 32-bit unsigned integer value of a datum.
- */
-static inline uint32
-DatumGetUInt32(Datum X)
-{
- return (uint32) X;
-}
-
-/*
- * UInt32GetDatum
- * Returns datum representation for a 32-bit unsigned integer.
- */
-static inline Datum
-UInt32GetDatum(uint32 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetObjectId
- * Returns object identifier value of a datum.
- */
-static inline Oid
-DatumGetObjectId(Datum X)
-{
- return (Oid) X;
-}
-
-/*
- * ObjectIdGetDatum
- * Returns datum representation for an object identifier.
- */
-static inline Datum
-ObjectIdGetDatum(Oid X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetTransactionId
- * Returns transaction identifier value of a datum.
- */
-static inline TransactionId
-DatumGetTransactionId(Datum X)
-{
- return (TransactionId) X;
-}
-
-/*
- * TransactionIdGetDatum
- * Returns datum representation for a transaction identifier.
- */
-static inline Datum
-TransactionIdGetDatum(TransactionId X)
-{
- return (Datum) X;
-}
-
-/*
- * MultiXactIdGetDatum
- * Returns datum representation for a multixact identifier.
- */
-static inline Datum
-MultiXactIdGetDatum(MultiXactId X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetCommandId
- * Returns command identifier value of a datum.
- */
-static inline CommandId
-DatumGetCommandId(Datum X)
-{
- return (CommandId) X;
-}
-
-/*
- * CommandIdGetDatum
- * Returns datum representation for a command identifier.
- */
-static inline Datum
-CommandIdGetDatum(CommandId X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetPointer
- * Returns pointer value of a datum.
- */
-static inline Pointer
-DatumGetPointer(Datum X)
-{
- return (Pointer) X;
-}
-
-/*
- * PointerGetDatum
- * Returns datum representation for a pointer.
- */
-static inline Datum
-PointerGetDatum(const void *X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetCString
- * Returns C string (null-terminated string) value of a datum.
- *
- * Note: C string is not a full-fledged Postgres type at present,
- * but type input functions use this conversion for their inputs.
- */
-static inline char *
-DatumGetCString(Datum X)
-{
- return (char *) DatumGetPointer(X);
-}
-
-/*
- * CStringGetDatum
- * Returns datum representation for a C string (null-terminated string).
- *
- * Note: C string is not a full-fledged Postgres type at present,
- * but type output functions use this conversion for their outputs.
- * Note: CString is pass-by-reference; caller must ensure the pointed-to
- * value has adequate lifetime.
- */
-static inline Datum
-CStringGetDatum(const char *X)
-{
- return PointerGetDatum(X);
-}
-
-/*
- * DatumGetName
- * Returns name value of a datum.
- */
-static inline Name
-DatumGetName(Datum X)
-{
- return (Name) DatumGetPointer(X);
-}
-
-/*
- * NameGetDatum
- * Returns datum representation for a name.
- *
- * Note: Name is pass-by-reference; caller must ensure the pointed-to
- * value has adequate lifetime.
- */
-static inline Datum
-NameGetDatum(const NameData *X)
-{
- return CStringGetDatum(NameStr(*X));
-}
-
-/*
- * DatumGetInt64
- * Returns 64-bit integer value of a datum.
- *
- * Note: this function hides whether int64 is pass by value or by reference.
- */
-static inline int64
-DatumGetInt64(Datum X)
-{
-#ifdef USE_FLOAT8_BYVAL
- return (int64) X;
-#else
- return *((int64 *) DatumGetPointer(X));
#endif
-}
-
-/*
- * Int64GetDatum
- * Returns datum representation for a 64-bit integer.
- *
- * Note: if int64 is pass by reference, this function returns a reference
- * to palloc'd space.
- */
-#ifdef USE_FLOAT8_BYVAL
-static inline Datum
-Int64GetDatum(int64 X)
-{
- return (Datum) X;
-}
-#else
-extern Datum Int64GetDatum(int64 X);
-#endif
-
-
-/*
- * DatumGetUInt64
- * Returns 64-bit unsigned integer value of a datum.
- *
- * Note: this function hides whether int64 is pass by value or by reference.
- */
-static inline uint64
-DatumGetUInt64(Datum X)
-{
-#ifdef USE_FLOAT8_BYVAL
- return (uint64) X;
-#else
- return *((uint64 *) DatumGetPointer(X));
-#endif
-}
-
-/*
- * UInt64GetDatum
- * Returns datum representation for a 64-bit unsigned integer.
- *
- * Note: if int64 is pass by reference, this function returns a reference
- * to palloc'd space.
- */
-static inline Datum
-UInt64GetDatum(uint64 X)
-{
-#ifdef USE_FLOAT8_BYVAL
- return (Datum) X;
-#else
- return Int64GetDatum((int64) X);
-#endif
-}
-
-/*
- * Float <-> Datum conversions
- *
- * These have to be implemented as inline functions rather than macros, when
- * passing by value, because many machines pass int and float function
- * parameters/results differently; so we need to play weird games with unions.
- */
-
-/*
- * DatumGetFloat4
- * Returns 4-byte floating point value of a datum.
- */
-static inline float4
-DatumGetFloat4(Datum X)
-{
- union
- {
- int32 value;
- float4 retval;
- } myunion;
-
- myunion.value = DatumGetInt32(X);
- return myunion.retval;
-}
-
-/*
- * Float4GetDatum
- * Returns datum representation for a 4-byte floating point number.
- */
-static inline Datum
-Float4GetDatum(float4 X)
-{
- union
- {
- float4 value;
- int32 retval;
- } myunion;
-
- myunion.value = X;
- return Int32GetDatum(myunion.retval);
-}
-
-/*
- * DatumGetFloat8
- * Returns 8-byte floating point value of a datum.
- *
- * Note: this function hides whether float8 is pass by value or by reference.
- */
-static inline float8
-DatumGetFloat8(Datum X)
-{
-#ifdef USE_FLOAT8_BYVAL
- union
- {
- int64 value;
- float8 retval;
- } myunion;
-
- myunion.value = DatumGetInt64(X);
- return myunion.retval;
-#else
- return *((float8 *) DatumGetPointer(X));
-#endif
-}
-
-/*
- * Float8GetDatum
- * Returns datum representation for an 8-byte floating point number.
- *
- * Note: if float8 is pass by reference, this function returns a reference
- * to palloc'd space.
- */
-#ifdef USE_FLOAT8_BYVAL
-static inline Datum
-Float8GetDatum(float8 X)
-{
- union
- {
- float8 value;
- int64 retval;
- } myunion;
-
- myunion.value = X;
- return Int64GetDatum(myunion.retval);
-}
-#else
-extern Datum Float8GetDatum(float8 X);
-#endif
-
-
-/*
- * Int64GetDatumFast
- * Float8GetDatumFast
- *
- * These macros are intended to allow writing code that does not depend on
- * whether int64 and float8 are pass-by-reference types, while not
- * sacrificing performance when they are. The argument must be a variable
- * that will exist and have the same value for as long as the Datum is needed.
- * In the pass-by-ref case, the address of the variable is taken to use as
- * the Datum. In the pass-by-val case, these are the same as the non-Fast
- * functions, except for asserting that the variable is of the correct type.
- */
-
-#ifdef USE_FLOAT8_BYVAL
-#define Int64GetDatumFast(X) \
- (AssertVariableIsOfTypeMacro(X, int64), Int64GetDatum(X))
-#define Float8GetDatumFast(X) \
- (AssertVariableIsOfTypeMacro(X, double), Float8GetDatum(X))
-#else
-#define Int64GetDatumFast(X) \
- (AssertVariableIsOfTypeMacro(X, int64), PointerGetDatum(&(X)))
-#define Float8GetDatumFast(X) \
- (AssertVariableIsOfTypeMacro(X, double), PointerGetDatum(&(X)))
-#endif
-
-
-/* ----------------------------------------------------------------
- * Section 3: miscellaneous
- * ----------------------------------------------------------------
- */
-
-/*
- * NON_EXEC_STATIC: It's sometimes useful to define a variable or function
- * that is normally static but extern when using EXEC_BACKEND (see
- * pg_config_manual.h). There would then typically be some code in
- * postmaster.c that uses those extern symbols to transfer state between
- * processes or do whatever other things it needs to do in EXEC_BACKEND mode.
- */
-#ifdef EXEC_BACKEND
-#define NON_EXEC_STATIC
-#else
-#define NON_EXEC_STATIC static
-#endif
-
-#endif /* POSTGRES_H */
base-commit: adb5c32eb53e1ffdc5c954aafcc5bc9ed93f3de6
--
2.39.0
On Wed, 28 Dec 2022 at 08:07, Peter Eisentraut <
peter.eisentraut@enterprisedb.com> wrote:
Most backend code doesn't actually need the variable-length data types
support (TOAST support) in postgres.h. So I figured we could try to put
it into a separate header file. That makes postgres.h more manageable,
and it avoids including a bunch of complicated unused stuff everywhere.
I picked "varatt.h" as the name. Then we could either
[…]
I went with the last option in my patch.
Thoughts?
This is a bit of a bikeshed suggestion, but I'm wondering if you considered
calling it toast.h? Only because the word is so distinctive within
Postgres; everybody knows exactly to what it refers.
I definitely agree with the principle of organizing and splitting up the
header files. Personally, I don't mind importing a bunch of headers if I'm
using a bunch of subsystems so I would be OK with needing to import this
new header if I need it.
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
... Then we could either
1) Include varatt.h in postgres.h, similar to elog.h and palloc.h. That
way we clean up the files a bit but don't change any external interfaces.
2) Just let everyone who needs it include the new file.
3) Compromise: You can avoid most "damage" by having fmgr.h include
varatt.h. That satisfies most data types and extension code. That way,
there are only a few places that need an explicit include of varatt.h.
I went with the last option in my patch.
I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included
so widely, I doubt it is buying very much in terms of reducing header
footprint. How bad is it to do #2?
regards, tom lane
Hi,
I've thought about this while working on Pluggable TOAST and 64-bit
TOAST value ID myself. Agree with #2 to seem the best of the above.
There are not so many places where a new header should be included.
On Wed, Dec 28, 2022 at 6:08 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
... Then we could either
1) Include varatt.h in postgres.h, similar to elog.h and palloc.h. That
way we clean up the files a bit but don't change any external interfaces.2) Just let everyone who needs it include the new file.
3) Compromise: You can avoid most "damage" by having fmgr.h include
varatt.h. That satisfies most data types and extension code. That way,
there are only a few places that need an explicit include of varatt.h.I went with the last option in my patch.
I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included
so widely, I doubt it is buying very much in terms of reducing header
footprint. How bad is it to do #2?regards, tom lane
--
Regards,
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/
Hi,
On 2022-12-28 09:07:12 -0500, Isaac Morland wrote:
This is a bit of a bikeshed suggestion, but I'm wondering if you considered
calling it toast.h? Only because the word is so distinctive within
Postgres; everybody knows exactly to what it refers.
We have a bunch of toast*.h files already. The new header should pretty much
only contain the types, given how widely the header is going to be
included. So maybe toast_type.h?
Greetings,
Andres Freund
On Thu, 29 Dec 2022 at 18:16, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2022-12-28 09:07:12 -0500, Isaac Morland wrote:
This is a bit of a bikeshed suggestion, but I'm wondering if you considered
calling it toast.h? Only because the word is so distinctive within
Postgres; everybody knows exactly to what it refers.We have a bunch of toast*.h files already. The new header should pretty much
only contain the types, given how widely the header is going to be
included. So maybe toast_type.h?
My 2 cents: I don't think that toast_anything.h is appropriate,
because even though the varatt infrastructure does enable
externally-stored oversized attributes (which is the essence of
TOAST), this is not the only (or primary) use of the type.
Example: Indexes do not (can not?) support toasted values, but
generally do support variable length attributes that would be pulled
in with varatt.h. I don't see why we'd call the headers of
variable-length attributes after one small - but not insignifcant -
use case.
Kind regards,
Matthias van de Meent
Matthias van de Meent <boekewurm+postgres@gmail.com> writes:
On Thu, 29 Dec 2022 at 18:16, Andres Freund <andres@anarazel.de> wrote:
We have a bunch of toast*.h files already. The new header should pretty much
only contain the types, given how widely the header is going to be
included. So maybe toast_type.h?
My 2 cents: I don't think that toast_anything.h is appropriate,
because even though the varatt infrastructure does enable
externally-stored oversized attributes (which is the essence of
TOAST), this is not the only (or primary) use of the type.
+1 ... varatt.h sounded fine to me. I'd suggest varlena.h except
we have one already.
regards, tom lane
On 28.12.22 16:07, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
... Then we could either
1) Include varatt.h in postgres.h, similar to elog.h and palloc.h. That
way we clean up the files a bit but don't change any external interfaces.2) Just let everyone who needs it include the new file.
3) Compromise: You can avoid most "damage" by having fmgr.h include
varatt.h. That satisfies most data types and extension code. That way,
there are only a few places that need an explicit include of varatt.h.I went with the last option in my patch.
I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included
so widely, I doubt it is buying very much in terms of reducing header
footprint. How bad is it to do #2?
See this incremental patch set.
It seems like maybe there is some intermediate abstraction that a lot of
these places should be using that we haven't thought of yet.
Attachments:
v2-0001-New-header-varatt.h-split-off-from-postgres.h.patchtext/plain; charset=UTF-8; name=v2-0001-New-header-varatt.h-split-off-from-postgres.h.patchDownload
From 04a1b66c9950991fb8dab141888c85ddd2d1a67c Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Wed, 28 Dec 2022 13:30:05 +0100
Subject: [PATCH v2 1/2] New header varatt.h split off from postgres.h
This new header contains all the variable-length data types support
(TOAST support) from postgres.h, which isn't needed by large parts of
the backend code.
This new header file is included by fmgr.h, so most data type and
extension code won't need any changes.
---
contrib/cube/cubedata.h | 2 +
contrib/pg_trgm/trgm_regexp.c | 1 +
src/backend/access/table/toast_helper.c | 1 +
src/backend/libpq/pqformat.c | 1 +
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 2 +
src/include/meson.build | 1 +
src/include/postgres.h | 358 +--------------
src/include/utils/expandeddatum.h | 2 +
src/include/{postgres.h => varatt.h} | 579 +-----------------------
10 files changed, 21 insertions(+), 927 deletions(-)
copy src/include/{postgres.h => varatt.h} (53%)
diff --git a/contrib/cube/cubedata.h b/contrib/cube/cubedata.h
index 96fa41a04e..d8df478059 100644
--- a/contrib/cube/cubedata.h
+++ b/contrib/cube/cubedata.h
@@ -1,5 +1,7 @@
/* contrib/cube/cubedata.h */
+#include "fmgr.h"
+
/*
* This limit is pretty arbitrary, but don't make it so large that you
* risk overflow in sizing calculations.
diff --git a/contrib/pg_trgm/trgm_regexp.c b/contrib/pg_trgm/trgm_regexp.c
index 58d32ba946..08eecd269b 100644
--- a/contrib/pg_trgm/trgm_regexp.c
+++ b/contrib/pg_trgm/trgm_regexp.c
@@ -196,6 +196,7 @@
#include "tsearch/ts_locale.h"
#include "utils/hsearch.h"
#include "utils/memutils.h"
+#include "varatt.h"
/*
* Uncomment (or use -DTRGM_REGEXP_DEBUG) to print debug info,
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 74ba2189f0..a4889d7642 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -19,6 +19,7 @@
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
+#include "varatt.h"
/*
diff --git a/src/backend/libpq/pqformat.c b/src/backend/libpq/pqformat.c
index 9c24df3360..65a913e893 100644
--- a/src/backend/libpq/pqformat.c
+++ b/src/backend/libpq/pqformat.c
@@ -77,6 +77,7 @@
#include "libpq/pqformat.h"
#include "mb/pg_wchar.h"
#include "port/pg_bswap.h"
+#include "varatt.h"
/* --------------------------------
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index c1af814e8d..c7b505e33f 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -19,6 +19,7 @@
#include "access/tupdesc.h"
#include "access/tupmacs.h"
#include "storage/bufpage.h"
+#include "varatt.h"
/*
* MaxTupleAttributeNumber limits the number of (user) columns in a tuple.
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index 972afe3aff..0ce0090360 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -18,6 +18,8 @@
#ifndef FMGR_H
#define FMGR_H
+#include "varatt.h"
+
/* We don't want to include primnodes.h here, so make some stub references */
typedef struct Node *fmNodePtr;
typedef struct Aggref *fmAggrefPtr;
diff --git a/src/include/meson.build b/src/include/meson.build
index b4820049c8..63b444bc99 100644
--- a/src/include/meson.build
+++ b/src/include/meson.build
@@ -113,6 +113,7 @@ install_headers(
'postgres.h',
'postgres_ext.h',
'postgres_fe.h',
+ 'varatt.h',
'windowapi.h',
pg_config_ext,
pg_config_os,
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 54730dfb38..a915f2b97f 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -23,9 +23,8 @@
*
* section description
* ------- ------------------------------------------------
- * 1) variable-length datatypes (TOAST support)
- * 2) Datum type + support functions
- * 3) miscellaneous
+ * 1) Datum type + support functions
+ * 2) miscellaneous
*
* NOTES
*
@@ -36,8 +35,7 @@
* postgres_fe.h. We do that since those type definitions are needed by
* frontend modules that want to deal with binary data transmission to or
* from the backend. Type definitions in this file should be for
- * representations that never escape the backend, such as Datum or
- * TOASTed varlena objects.
+ * representations that never escape the backend, such as Datum.
*
*----------------------------------------------------------------
*/
@@ -49,353 +47,7 @@
#include "utils/palloc.h"
/* ----------------------------------------------------------------
- * Section 1: variable-length datatypes (TOAST support)
- * ----------------------------------------------------------------
- */
-
-/*
- * struct varatt_external is a traditional "TOAST pointer", that is, the
- * information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if the external size stored in
- * va_extinfo is less than va_rawsize - VARHDRSZ.
- *
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
- *
- * Note that this information is stored unaligned within actual tuples, so
- * you need to memcpy from the tuple into a local struct variable before
- * you can look at these fields! (The reason we use memcmp is to avoid
- * having to do that just to detect equality of two TOAST pointers...)
- */
-typedef struct varatt_external
-{
- int32 va_rawsize; /* Original data size (includes header) */
- uint32 va_extinfo; /* External saved size (without header) and
- * compression method */
- Oid va_valueid; /* Unique ID of value within TOAST table */
- Oid va_toastrelid; /* RelID of TOAST table containing it */
-} varatt_external;
-
-/*
- * These macros define the "saved size" portion of va_extinfo. Its remaining
- * two high-order bits identify the compression method.
- */
-#define VARLENA_EXTSIZE_BITS 30
-#define VARLENA_EXTSIZE_MASK ((1U << VARLENA_EXTSIZE_BITS) - 1)
-
-/*
- * struct varatt_indirect is a "TOAST pointer" representing an out-of-line
- * Datum that's stored in memory, not in an external toast relation.
- * The creator of such a Datum is entirely responsible that the referenced
- * storage survives for as long as referencing pointer Datums can exist.
- *
- * Note that just as for struct varatt_external, this struct is stored
- * unaligned within any containing tuple.
- */
-typedef struct varatt_indirect
-{
- struct varlena *pointer; /* Pointer to in-memory varlena */
-} varatt_indirect;
-
-/*
- * struct varatt_expanded is a "TOAST pointer" representing an out-of-line
- * Datum that is stored in memory, in some type-specific, not necessarily
- * physically contiguous format that is convenient for computation not
- * storage. APIs for this, in particular the definition of struct
- * ExpandedObjectHeader, are in src/include/utils/expandeddatum.h.
- *
- * Note that just as for struct varatt_external, this struct is stored
- * unaligned within any containing tuple.
- */
-typedef struct ExpandedObjectHeader ExpandedObjectHeader;
-
-typedef struct varatt_expanded
-{
- ExpandedObjectHeader *eohptr;
-} varatt_expanded;
-
-/*
- * Type tag for the various sorts of "TOAST pointer" datums. The peculiar
- * value for VARTAG_ONDISK comes from a requirement for on-disk compatibility
- * with a previous notion that the tag field was the pointer datum's length.
- */
-typedef enum vartag_external
-{
- VARTAG_INDIRECT = 1,
- VARTAG_EXPANDED_RO = 2,
- VARTAG_EXPANDED_RW = 3,
- VARTAG_ONDISK = 18
-} vartag_external;
-
-/* this test relies on the specific tag values above */
-#define VARTAG_IS_EXPANDED(tag) \
- (((tag) & ~1) == VARTAG_EXPANDED_RO)
-
-#define VARTAG_SIZE(tag) \
- ((tag) == VARTAG_INDIRECT ? sizeof(varatt_indirect) : \
- VARTAG_IS_EXPANDED(tag) ? sizeof(varatt_expanded) : \
- (tag) == VARTAG_ONDISK ? sizeof(varatt_external) : \
- (AssertMacro(false), 0))
-
-/*
- * These structs describe the header of a varlena object that may have been
- * TOASTed. Generally, don't reference these structs directly, but use the
- * macros below.
- *
- * We use separate structs for the aligned and unaligned cases because the
- * compiler might otherwise think it could generate code that assumes
- * alignment while touching fields of a 1-byte-header varlena.
- */
-typedef union
-{
- struct /* Normal varlena (4-byte length) */
- {
- uint32 va_header;
- char va_data[FLEXIBLE_ARRAY_MEMBER];
- } va_4byte;
- struct /* Compressed-in-line format */
- {
- uint32 va_header;
- uint32 va_tcinfo; /* Original data size (excludes header) and
- * compression method; see va_extinfo */
- char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
- } va_compressed;
-} varattrib_4b;
-
-typedef struct
-{
- uint8 va_header;
- char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Data begins here */
-} varattrib_1b;
-
-/* TOAST pointers are a subset of varattrib_1b with an identifying tag byte */
-typedef struct
-{
- uint8 va_header; /* Always 0x80 or 0x01 */
- uint8 va_tag; /* Type of datum */
- char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Type-specific data */
-} varattrib_1b_e;
-
-/*
- * Bit layouts for varlena headers on big-endian machines:
- *
- * 00xxxxxx 4-byte length word, aligned, uncompressed data (up to 1G)
- * 01xxxxxx 4-byte length word, aligned, *compressed* data (up to 1G)
- * 10000000 1-byte length word, unaligned, TOAST pointer
- * 1xxxxxxx 1-byte length word, unaligned, uncompressed data (up to 126b)
- *
- * Bit layouts for varlena headers on little-endian machines:
- *
- * xxxxxx00 4-byte length word, aligned, uncompressed data (up to 1G)
- * xxxxxx10 4-byte length word, aligned, *compressed* data (up to 1G)
- * 00000001 1-byte length word, unaligned, TOAST pointer
- * xxxxxxx1 1-byte length word, unaligned, uncompressed data (up to 126b)
- *
- * The "xxx" bits are the length field (which includes itself in all cases).
- * In the big-endian case we mask to extract the length, in the little-endian
- * case we shift. Note that in both cases the flag bits are in the physically
- * first byte. Also, it is not possible for a 1-byte length word to be zero;
- * this lets us disambiguate alignment padding bytes from the start of an
- * unaligned datum. (We now *require* pad bytes to be filled with zero!)
- *
- * In TOAST pointers the va_tag field (see varattrib_1b_e) is used to discern
- * the specific type and length of the pointer datum.
- */
-
-/*
- * Endian-dependent macros. These are considered internal --- use the
- * external macros below instead of using these directly.
- *
- * Note: IS_1B is true for external toast records but VARSIZE_1B will return 0
- * for such records. Hence you should usually check for IS_EXTERNAL before
- * checking for IS_1B.
- */
-
-#ifdef WORDS_BIGENDIAN
-
-#define VARATT_IS_4B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x80) == 0x00)
-#define VARATT_IS_4B_U(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0xC0) == 0x00)
-#define VARATT_IS_4B_C(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0xC0) == 0x40)
-#define VARATT_IS_1B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x80) == 0x80)
-#define VARATT_IS_1B_E(PTR) \
- ((((varattrib_1b *) (PTR))->va_header) == 0x80)
-#define VARATT_NOT_PAD_BYTE(PTR) \
- (*((uint8 *) (PTR)) != 0)
-
-/* VARSIZE_4B() should only be used on known-aligned data */
-#define VARSIZE_4B(PTR) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header & 0x3FFFFFFF)
-#define VARSIZE_1B(PTR) \
- (((varattrib_1b *) (PTR))->va_header & 0x7F)
-#define VARTAG_1B_E(PTR) \
- (((varattrib_1b_e *) (PTR))->va_tag)
-
-#define SET_VARSIZE_4B(PTR,len) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header = (len) & 0x3FFFFFFF)
-#define SET_VARSIZE_4B_C(PTR,len) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header = ((len) & 0x3FFFFFFF) | 0x40000000)
-#define SET_VARSIZE_1B(PTR,len) \
- (((varattrib_1b *) (PTR))->va_header = (len) | 0x80)
-#define SET_VARTAG_1B_E(PTR,tag) \
- (((varattrib_1b_e *) (PTR))->va_header = 0x80, \
- ((varattrib_1b_e *) (PTR))->va_tag = (tag))
-
-#else /* !WORDS_BIGENDIAN */
-
-#define VARATT_IS_4B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x01) == 0x00)
-#define VARATT_IS_4B_U(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x03) == 0x00)
-#define VARATT_IS_4B_C(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x03) == 0x02)
-#define VARATT_IS_1B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header & 0x01) == 0x01)
-#define VARATT_IS_1B_E(PTR) \
- ((((varattrib_1b *) (PTR))->va_header) == 0x01)
-#define VARATT_NOT_PAD_BYTE(PTR) \
- (*((uint8 *) (PTR)) != 0)
-
-/* VARSIZE_4B() should only be used on known-aligned data */
-#define VARSIZE_4B(PTR) \
- ((((varattrib_4b *) (PTR))->va_4byte.va_header >> 2) & 0x3FFFFFFF)
-#define VARSIZE_1B(PTR) \
- ((((varattrib_1b *) (PTR))->va_header >> 1) & 0x7F)
-#define VARTAG_1B_E(PTR) \
- (((varattrib_1b_e *) (PTR))->va_tag)
-
-#define SET_VARSIZE_4B(PTR,len) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header = (((uint32) (len)) << 2))
-#define SET_VARSIZE_4B_C(PTR,len) \
- (((varattrib_4b *) (PTR))->va_4byte.va_header = (((uint32) (len)) << 2) | 0x02)
-#define SET_VARSIZE_1B(PTR,len) \
- (((varattrib_1b *) (PTR))->va_header = (((uint8) (len)) << 1) | 0x01)
-#define SET_VARTAG_1B_E(PTR,tag) \
- (((varattrib_1b_e *) (PTR))->va_header = 0x01, \
- ((varattrib_1b_e *) (PTR))->va_tag = (tag))
-
-#endif /* WORDS_BIGENDIAN */
-
-#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
-#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
-#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
-#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
-
-/*
- * Externally visible TOAST macros begin here.
- */
-
-#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
-#define VARHDRSZ_COMPRESSED offsetof(varattrib_4b, va_compressed.va_data)
-#define VARHDRSZ_SHORT offsetof(varattrib_1b, va_data)
-
-#define VARATT_SHORT_MAX 0x7F
-#define VARATT_CAN_MAKE_SHORT(PTR) \
- (VARATT_IS_4B_U(PTR) && \
- (VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT) <= VARATT_SHORT_MAX)
-#define VARATT_CONVERTED_SHORT_SIZE(PTR) \
- (VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
-
-/*
- * In consumers oblivious to data alignment, call PG_DETOAST_DATUM_PACKED(),
- * VARDATA_ANY(), VARSIZE_ANY() and VARSIZE_ANY_EXHDR(). Elsewhere, call
- * PG_DETOAST_DATUM(), VARDATA() and VARSIZE(). Directly fetching an int16,
- * int32 or wider field in the struct representing the datum layout requires
- * aligned data. memcpy() is alignment-oblivious, as are most operations on
- * datatypes, such as text, whose layout struct contains only char fields.
- *
- * Code assembling a new datum should call VARDATA() and SET_VARSIZE().
- * (Datums begin life untoasted.)
- *
- * Other macros here should usually be used only by tuple assembly/disassembly
- * code and code that specifically wants to work with still-toasted Datums.
- */
-#define VARDATA(PTR) VARDATA_4B(PTR)
-#define VARSIZE(PTR) VARSIZE_4B(PTR)
-
-#define VARSIZE_SHORT(PTR) VARSIZE_1B(PTR)
-#define VARDATA_SHORT(PTR) VARDATA_1B(PTR)
-
-#define VARTAG_EXTERNAL(PTR) VARTAG_1B_E(PTR)
-#define VARSIZE_EXTERNAL(PTR) (VARHDRSZ_EXTERNAL + VARTAG_SIZE(VARTAG_EXTERNAL(PTR)))
-#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
-
-#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
-#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
-#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
-#define VARATT_IS_EXTERNAL_INDIRECT(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_INDIRECT)
-#define VARATT_IS_EXTERNAL_EXPANDED_RO(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_EXPANDED_RO)
-#define VARATT_IS_EXTERNAL_EXPANDED_RW(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_EXPANDED_RW)
-#define VARATT_IS_EXTERNAL_EXPANDED(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
-#define VARATT_IS_EXTERNAL_NON_EXPANDED(PTR) \
- (VARATT_IS_EXTERNAL(PTR) && !VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
-#define VARATT_IS_SHORT(PTR) VARATT_IS_1B(PTR)
-#define VARATT_IS_EXTENDED(PTR) (!VARATT_IS_4B_U(PTR))
-
-#define SET_VARSIZE(PTR, len) SET_VARSIZE_4B(PTR, len)
-#define SET_VARSIZE_SHORT(PTR, len) SET_VARSIZE_1B(PTR, len)
-#define SET_VARSIZE_COMPRESSED(PTR, len) SET_VARSIZE_4B_C(PTR, len)
-
-#define SET_VARTAG_EXTERNAL(PTR, tag) SET_VARTAG_1B_E(PTR, tag)
-
-#define VARSIZE_ANY(PTR) \
- (VARATT_IS_1B_E(PTR) ? VARSIZE_EXTERNAL(PTR) : \
- (VARATT_IS_1B(PTR) ? VARSIZE_1B(PTR) : \
- VARSIZE_4B(PTR)))
-
-/* Size of a varlena data, excluding header */
-#define VARSIZE_ANY_EXHDR(PTR) \
- (VARATT_IS_1B_E(PTR) ? VARSIZE_EXTERNAL(PTR)-VARHDRSZ_EXTERNAL : \
- (VARATT_IS_1B(PTR) ? VARSIZE_1B(PTR)-VARHDRSZ_SHORT : \
- VARSIZE_4B(PTR)-VARHDRSZ))
-
-/* caution: this will not work on an external or compressed-in-line Datum */
-/* caution: this will return a possibly unaligned pointer */
-#define VARDATA_ANY(PTR) \
- (VARATT_IS_1B(PTR) ? VARDATA_1B(PTR) : VARDATA_4B(PTR))
-
-/* Decompressed size and compression method of a compressed-in-line Datum */
-#define VARDATA_COMPRESSED_GET_EXTSIZE(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_EXTSIZE_MASK)
-#define VARDATA_COMPRESSED_GET_COMPRESS_METHOD(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS)
-
-/* Same for external Datums; but note argument is a struct varatt_external */
-#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
- ((toast_pointer).va_extinfo & VARLENA_EXTSIZE_MASK)
-#define VARATT_EXTERNAL_GET_COMPRESS_METHOD(toast_pointer) \
- ((toast_pointer).va_extinfo >> VARLENA_EXTSIZE_BITS)
-
-#define VARATT_EXTERNAL_SET_SIZE_AND_COMPRESS_METHOD(toast_pointer, len, cm) \
- do { \
- Assert((cm) == TOAST_PGLZ_COMPRESSION_ID || \
- (cm) == TOAST_LZ4_COMPRESSION_ID); \
- ((toast_pointer).va_extinfo = \
- (len) | ((uint32) (cm) << VARLENA_EXTSIZE_BITS)); \
- } while (0)
-
-/*
- * Testing whether an externally-stored value is compressed now requires
- * comparing size stored in va_extinfo (the actual length of the external data)
- * to rawsize (the original uncompressed datum's size). The latter includes
- * VARHDRSZ overhead, the former doesn't. We never use compression unless it
- * actually saves space, so we expect either equality or less-than.
- */
-#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < \
- (toast_pointer).va_rawsize - VARHDRSZ)
-
-
-/* ----------------------------------------------------------------
- * Section 2: Datum type + support functions
+ * Section 1: Datum type + support functions
* ----------------------------------------------------------------
*/
@@ -907,7 +559,7 @@ extern Datum Float8GetDatum(float8 X);
/* ----------------------------------------------------------------
- * Section 3: miscellaneous
+ * Section 2: miscellaneous
* ----------------------------------------------------------------
*/
diff --git a/src/include/utils/expandeddatum.h b/src/include/utils/expandeddatum.h
index c4818eb054..139f49d940 100644
--- a/src/include/utils/expandeddatum.h
+++ b/src/include/utils/expandeddatum.h
@@ -44,6 +44,8 @@
#ifndef EXPANDEDDATUM_H
#define EXPANDEDDATUM_H
+#include "varatt.h"
+
/* Size of an EXTERNAL datum that contains a pointer to an expanded object */
#define EXPANDED_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_expanded))
diff --git a/src/include/postgres.h b/src/include/varatt.h
similarity index 53%
copy from src/include/postgres.h
copy to src/include/varatt.h
index 54730dfb38..37a139145e 100644
--- a/src/include/postgres.h
+++ b/src/include/varatt.h
@@ -1,57 +1,19 @@
/*-------------------------------------------------------------------------
*
- * postgres.h
- * Primary include file for PostgreSQL server .c files
- *
- * This should be the first file included by PostgreSQL backend modules.
- * Client-side code should include postgres_fe.h instead.
+ * varatt.h
+ * variable-length datatypes (TOAST support)
*
*
* Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
* Portions Copyright (c) 1995, Regents of the University of California
*
- * src/include/postgres.h
+ * src/include/varatt.h
*
*-------------------------------------------------------------------------
*/
-/*
- *----------------------------------------------------------------
- * TABLE OF CONTENTS
- *
- * When adding stuff to this file, please try to put stuff
- * into the relevant section, or add new sections as appropriate.
- *
- * section description
- * ------- ------------------------------------------------
- * 1) variable-length datatypes (TOAST support)
- * 2) Datum type + support functions
- * 3) miscellaneous
- *
- * NOTES
- *
- * In general, this file should contain declarations that are widely needed
- * in the backend environment, but are of no interest outside the backend.
- *
- * Simple type definitions live in c.h, where they are shared with
- * postgres_fe.h. We do that since those type definitions are needed by
- * frontend modules that want to deal with binary data transmission to or
- * from the backend. Type definitions in this file should be for
- * representations that never escape the backend, such as Datum or
- * TOASTed varlena objects.
- *
- *----------------------------------------------------------------
- */
-#ifndef POSTGRES_H
-#define POSTGRES_H
-#include "c.h"
-#include "utils/elog.h"
-#include "utils/palloc.h"
-
-/* ----------------------------------------------------------------
- * Section 1: variable-length datatypes (TOAST support)
- * ----------------------------------------------------------------
- */
+#ifndef VARATT_H
+#define VARATT_H
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
@@ -393,535 +355,4 @@ typedef struct
(VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < \
(toast_pointer).va_rawsize - VARHDRSZ)
-
-/* ----------------------------------------------------------------
- * Section 2: Datum type + support functions
- * ----------------------------------------------------------------
- */
-
-/*
- * A Datum contains either a value of a pass-by-value type or a pointer to a
- * value of a pass-by-reference type. Therefore, we require:
- *
- * sizeof(Datum) == sizeof(void *) == 4 or 8
- *
- * The functions below and the analogous functions for other types should be used to
- * convert between a Datum and the appropriate C type.
- */
-
-typedef uintptr_t Datum;
-
-/*
- * A NullableDatum is used in places where both a Datum and its nullness needs
- * to be stored. This can be more efficient than storing datums and nullness
- * in separate arrays, due to better spatial locality, even if more space may
- * be wasted due to padding.
- */
-typedef struct NullableDatum
-{
-#define FIELDNO_NULLABLE_DATUM_DATUM 0
- Datum value;
-#define FIELDNO_NULLABLE_DATUM_ISNULL 1
- bool isnull;
- /* due to alignment padding this could be used for flags for free */
-} NullableDatum;
-
-#define SIZEOF_DATUM SIZEOF_VOID_P
-
-/*
- * DatumGetBool
- * Returns boolean value of a datum.
- *
- * Note: any nonzero value will be considered true.
- */
-static inline bool
-DatumGetBool(Datum X)
-{
- return (X != 0);
-}
-
-/*
- * BoolGetDatum
- * Returns datum representation for a boolean.
- *
- * Note: any nonzero value will be considered true.
- */
-static inline Datum
-BoolGetDatum(bool X)
-{
- return (Datum) (X ? 1 : 0);
-}
-
-/*
- * DatumGetChar
- * Returns character value of a datum.
- */
-static inline char
-DatumGetChar(Datum X)
-{
- return (char) X;
-}
-
-/*
- * CharGetDatum
- * Returns datum representation for a character.
- */
-static inline Datum
-CharGetDatum(char X)
-{
- return (Datum) X;
-}
-
-/*
- * Int8GetDatum
- * Returns datum representation for an 8-bit integer.
- */
-static inline Datum
-Int8GetDatum(int8 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetUInt8
- * Returns 8-bit unsigned integer value of a datum.
- */
-static inline uint8
-DatumGetUInt8(Datum X)
-{
- return (uint8) X;
-}
-
-/*
- * UInt8GetDatum
- * Returns datum representation for an 8-bit unsigned integer.
- */
-static inline Datum
-UInt8GetDatum(uint8 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetInt16
- * Returns 16-bit integer value of a datum.
- */
-static inline int16
-DatumGetInt16(Datum X)
-{
- return (int16) X;
-}
-
-/*
- * Int16GetDatum
- * Returns datum representation for a 16-bit integer.
- */
-static inline Datum
-Int16GetDatum(int16 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetUInt16
- * Returns 16-bit unsigned integer value of a datum.
- */
-static inline uint16
-DatumGetUInt16(Datum X)
-{
- return (uint16) X;
-}
-
-/*
- * UInt16GetDatum
- * Returns datum representation for a 16-bit unsigned integer.
- */
-static inline Datum
-UInt16GetDatum(uint16 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetInt32
- * Returns 32-bit integer value of a datum.
- */
-static inline int32
-DatumGetInt32(Datum X)
-{
- return (int32) X;
-}
-
-/*
- * Int32GetDatum
- * Returns datum representation for a 32-bit integer.
- */
-static inline Datum
-Int32GetDatum(int32 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetUInt32
- * Returns 32-bit unsigned integer value of a datum.
- */
-static inline uint32
-DatumGetUInt32(Datum X)
-{
- return (uint32) X;
-}
-
-/*
- * UInt32GetDatum
- * Returns datum representation for a 32-bit unsigned integer.
- */
-static inline Datum
-UInt32GetDatum(uint32 X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetObjectId
- * Returns object identifier value of a datum.
- */
-static inline Oid
-DatumGetObjectId(Datum X)
-{
- return (Oid) X;
-}
-
-/*
- * ObjectIdGetDatum
- * Returns datum representation for an object identifier.
- */
-static inline Datum
-ObjectIdGetDatum(Oid X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetTransactionId
- * Returns transaction identifier value of a datum.
- */
-static inline TransactionId
-DatumGetTransactionId(Datum X)
-{
- return (TransactionId) X;
-}
-
-/*
- * TransactionIdGetDatum
- * Returns datum representation for a transaction identifier.
- */
-static inline Datum
-TransactionIdGetDatum(TransactionId X)
-{
- return (Datum) X;
-}
-
-/*
- * MultiXactIdGetDatum
- * Returns datum representation for a multixact identifier.
- */
-static inline Datum
-MultiXactIdGetDatum(MultiXactId X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetCommandId
- * Returns command identifier value of a datum.
- */
-static inline CommandId
-DatumGetCommandId(Datum X)
-{
- return (CommandId) X;
-}
-
-/*
- * CommandIdGetDatum
- * Returns datum representation for a command identifier.
- */
-static inline Datum
-CommandIdGetDatum(CommandId X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetPointer
- * Returns pointer value of a datum.
- */
-static inline Pointer
-DatumGetPointer(Datum X)
-{
- return (Pointer) X;
-}
-
-/*
- * PointerGetDatum
- * Returns datum representation for a pointer.
- */
-static inline Datum
-PointerGetDatum(const void *X)
-{
- return (Datum) X;
-}
-
-/*
- * DatumGetCString
- * Returns C string (null-terminated string) value of a datum.
- *
- * Note: C string is not a full-fledged Postgres type at present,
- * but type input functions use this conversion for their inputs.
- */
-static inline char *
-DatumGetCString(Datum X)
-{
- return (char *) DatumGetPointer(X);
-}
-
-/*
- * CStringGetDatum
- * Returns datum representation for a C string (null-terminated string).
- *
- * Note: C string is not a full-fledged Postgres type at present,
- * but type output functions use this conversion for their outputs.
- * Note: CString is pass-by-reference; caller must ensure the pointed-to
- * value has adequate lifetime.
- */
-static inline Datum
-CStringGetDatum(const char *X)
-{
- return PointerGetDatum(X);
-}
-
-/*
- * DatumGetName
- * Returns name value of a datum.
- */
-static inline Name
-DatumGetName(Datum X)
-{
- return (Name) DatumGetPointer(X);
-}
-
-/*
- * NameGetDatum
- * Returns datum representation for a name.
- *
- * Note: Name is pass-by-reference; caller must ensure the pointed-to
- * value has adequate lifetime.
- */
-static inline Datum
-NameGetDatum(const NameData *X)
-{
- return CStringGetDatum(NameStr(*X));
-}
-
-/*
- * DatumGetInt64
- * Returns 64-bit integer value of a datum.
- *
- * Note: this function hides whether int64 is pass by value or by reference.
- */
-static inline int64
-DatumGetInt64(Datum X)
-{
-#ifdef USE_FLOAT8_BYVAL
- return (int64) X;
-#else
- return *((int64 *) DatumGetPointer(X));
-#endif
-}
-
-/*
- * Int64GetDatum
- * Returns datum representation for a 64-bit integer.
- *
- * Note: if int64 is pass by reference, this function returns a reference
- * to palloc'd space.
- */
-#ifdef USE_FLOAT8_BYVAL
-static inline Datum
-Int64GetDatum(int64 X)
-{
- return (Datum) X;
-}
-#else
-extern Datum Int64GetDatum(int64 X);
#endif
-
-
-/*
- * DatumGetUInt64
- * Returns 64-bit unsigned integer value of a datum.
- *
- * Note: this function hides whether int64 is pass by value or by reference.
- */
-static inline uint64
-DatumGetUInt64(Datum X)
-{
-#ifdef USE_FLOAT8_BYVAL
- return (uint64) X;
-#else
- return *((uint64 *) DatumGetPointer(X));
-#endif
-}
-
-/*
- * UInt64GetDatum
- * Returns datum representation for a 64-bit unsigned integer.
- *
- * Note: if int64 is pass by reference, this function returns a reference
- * to palloc'd space.
- */
-static inline Datum
-UInt64GetDatum(uint64 X)
-{
-#ifdef USE_FLOAT8_BYVAL
- return (Datum) X;
-#else
- return Int64GetDatum((int64) X);
-#endif
-}
-
-/*
- * Float <-> Datum conversions
- *
- * These have to be implemented as inline functions rather than macros, when
- * passing by value, because many machines pass int and float function
- * parameters/results differently; so we need to play weird games with unions.
- */
-
-/*
- * DatumGetFloat4
- * Returns 4-byte floating point value of a datum.
- */
-static inline float4
-DatumGetFloat4(Datum X)
-{
- union
- {
- int32 value;
- float4 retval;
- } myunion;
-
- myunion.value = DatumGetInt32(X);
- return myunion.retval;
-}
-
-/*
- * Float4GetDatum
- * Returns datum representation for a 4-byte floating point number.
- */
-static inline Datum
-Float4GetDatum(float4 X)
-{
- union
- {
- float4 value;
- int32 retval;
- } myunion;
-
- myunion.value = X;
- return Int32GetDatum(myunion.retval);
-}
-
-/*
- * DatumGetFloat8
- * Returns 8-byte floating point value of a datum.
- *
- * Note: this function hides whether float8 is pass by value or by reference.
- */
-static inline float8
-DatumGetFloat8(Datum X)
-{
-#ifdef USE_FLOAT8_BYVAL
- union
- {
- int64 value;
- float8 retval;
- } myunion;
-
- myunion.value = DatumGetInt64(X);
- return myunion.retval;
-#else
- return *((float8 *) DatumGetPointer(X));
-#endif
-}
-
-/*
- * Float8GetDatum
- * Returns datum representation for an 8-byte floating point number.
- *
- * Note: if float8 is pass by reference, this function returns a reference
- * to palloc'd space.
- */
-#ifdef USE_FLOAT8_BYVAL
-static inline Datum
-Float8GetDatum(float8 X)
-{
- union
- {
- float8 value;
- int64 retval;
- } myunion;
-
- myunion.value = X;
- return Int64GetDatum(myunion.retval);
-}
-#else
-extern Datum Float8GetDatum(float8 X);
-#endif
-
-
-/*
- * Int64GetDatumFast
- * Float8GetDatumFast
- *
- * These macros are intended to allow writing code that does not depend on
- * whether int64 and float8 are pass-by-reference types, while not
- * sacrificing performance when they are. The argument must be a variable
- * that will exist and have the same value for as long as the Datum is needed.
- * In the pass-by-ref case, the address of the variable is taken to use as
- * the Datum. In the pass-by-val case, these are the same as the non-Fast
- * functions, except for asserting that the variable is of the correct type.
- */
-
-#ifdef USE_FLOAT8_BYVAL
-#define Int64GetDatumFast(X) \
- (AssertVariableIsOfTypeMacro(X, int64), Int64GetDatum(X))
-#define Float8GetDatumFast(X) \
- (AssertVariableIsOfTypeMacro(X, double), Float8GetDatum(X))
-#else
-#define Int64GetDatumFast(X) \
- (AssertVariableIsOfTypeMacro(X, int64), PointerGetDatum(&(X)))
-#define Float8GetDatumFast(X) \
- (AssertVariableIsOfTypeMacro(X, double), PointerGetDatum(&(X)))
-#endif
-
-
-/* ----------------------------------------------------------------
- * Section 3: miscellaneous
- * ----------------------------------------------------------------
- */
-
-/*
- * NON_EXEC_STATIC: It's sometimes useful to define a variable or function
- * that is normally static but extern when using EXEC_BACKEND (see
- * pg_config_manual.h). There would then typically be some code in
- * postmaster.c that uses those extern symbols to transfer state between
- * processes or do whatever other things it needs to do in EXEC_BACKEND mode.
- */
-#ifdef EXEC_BACKEND
-#define NON_EXEC_STATIC
-#else
-#define NON_EXEC_STATIC static
-#endif
-
-#endif /* POSTGRES_H */
--
2.39.0
v2-0002-Move-varatt.h-include-to-individual-files.patchtext/plain; charset=UTF-8; name=v2-0002-Move-varatt.h-include-to-individual-files.patchDownload
From cf18825445a486eb073d1f3d0b159b3f4fe57fb1 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Fri, 30 Dec 2022 12:08:21 +0100
Subject: [PATCH v2 2/2] Move varatt.h include to individual files
---
contrib/citext/citext.c | 1 +
contrib/cube/cubeparse.y | 1 +
contrib/fuzzystrmatch/fuzzystrmatch.c | 1 +
contrib/ltree/ltree_io.c | 1 +
contrib/ltree/ltxtquery_io.c | 1 +
contrib/pg_trgm/trgm_gin.c | 1 +
contrib/pg_trgm/trgm_gist.c | 1 +
contrib/pgcrypto/pgcrypto.c | 1 +
contrib/uuid-ossp/uuid-ossp.c | 1 +
src/backend/access/common/toast_compression.c | 1 +
src/backend/access/hash/hashfunc.c | 1 +
src/backend/access/spgist/spgtextproc.c | 1 +
src/backend/libpq/be-fsstubs.c | 1 +
src/backend/tsearch/ts_parse.c | 1 +
src/backend/tsearch/ts_typanalyze.c | 1 +
src/backend/utils/adt/ascii.c | 1 +
src/backend/utils/adt/char.c | 1 +
src/backend/utils/adt/cryptohashfuncs.c | 1 +
src/backend/utils/adt/encode.c | 1 +
src/backend/utils/adt/formatting.c | 1 +
src/backend/utils/adt/geo_ops.c | 1 +
src/backend/utils/adt/like.c | 1 +
src/backend/utils/adt/network_gist.c | 1 +
src/backend/utils/adt/network_spgist.c | 1 +
src/backend/utils/adt/oracle_compat.c | 1 +
src/backend/utils/adt/quote.c | 1 +
src/backend/utils/adt/rangetypes.c | 1 +
src/backend/utils/adt/rangetypes_typanalyze.c | 1 +
src/backend/utils/adt/tsginidx.c | 1 +
src/backend/utils/adt/tsquery.c | 1 +
src/backend/utils/adt/tsquery_cleanup.c | 1 +
src/backend/utils/adt/tsquery_op.c | 1 +
src/backend/utils/adt/tsquery_util.c | 1 +
src/backend/utils/adt/tsvector.c | 1 +
src/backend/utils/cache/attoptcache.c | 1 +
src/backend/utils/cache/spccache.c | 1 +
src/backend/utils/mb/mbutils.c | 1 +
src/include/fmgr.h | 2 --
src/pl/plperl/Util.xs | 1 +
src/test/modules/spgist_name_ops/spgist_name_ops.c | 1 +
src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c | 1 +
src/test/modules/test_shm_mq/test.c | 1 +
42 files changed, 41 insertions(+), 2 deletions(-)
diff --git a/contrib/citext/citext.c b/contrib/citext/citext.c
index df139462a6..976c578e1c 100644
--- a/contrib/citext/citext.c
+++ b/contrib/citext/citext.c
@@ -8,6 +8,7 @@
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/varlena.h"
+#include "varatt.h"
PG_MODULE_MAGIC;
diff --git a/contrib/cube/cubeparse.y b/contrib/cube/cubeparse.y
index 44450d1027..b39fbe63e6 100644
--- a/contrib/cube/cubeparse.y
+++ b/contrib/cube/cubeparse.y
@@ -9,6 +9,7 @@
#include "cubedata.h"
#include "nodes/miscnodes.h"
#include "utils/float.h"
+#include "varatt.h"
/* All grammar constructs return strings */
#define YYSTYPE char *
diff --git a/contrib/fuzzystrmatch/fuzzystrmatch.c b/contrib/fuzzystrmatch/fuzzystrmatch.c
index 5659cc71ac..cd19d6af62 100644
--- a/contrib/fuzzystrmatch/fuzzystrmatch.c
+++ b/contrib/fuzzystrmatch/fuzzystrmatch.c
@@ -43,6 +43,7 @@
#include "mb/pg_wchar.h"
#include "utils/builtins.h"
#include "utils/varlena.h"
+#include "varatt.h"
PG_MODULE_MAGIC;
diff --git a/contrib/ltree/ltree_io.c b/contrib/ltree/ltree_io.c
index f0dd3df511..f775cf7b04 100644
--- a/contrib/ltree/ltree_io.c
+++ b/contrib/ltree/ltree_io.c
@@ -11,6 +11,7 @@
#include "libpq/pqformat.h"
#include "ltree.h"
#include "utils/memutils.h"
+#include "varatt.h"
typedef struct
diff --git a/contrib/ltree/ltxtquery_io.c b/contrib/ltree/ltxtquery_io.c
index a16e577303..6e476ce9ea 100644
--- a/contrib/ltree/ltxtquery_io.c
+++ b/contrib/ltree/ltxtquery_io.c
@@ -12,6 +12,7 @@
#include "ltree.h"
#include "miscadmin.h"
#include "nodes/miscnodes.h"
+#include "varatt.h"
/* parser's states */
diff --git a/contrib/pg_trgm/trgm_gin.c b/contrib/pg_trgm/trgm_gin.c
index 32fafef203..29a52eac7a 100644
--- a/contrib/pg_trgm/trgm_gin.c
+++ b/contrib/pg_trgm/trgm_gin.c
@@ -7,6 +7,7 @@
#include "access/stratnum.h"
#include "fmgr.h"
#include "trgm.h"
+#include "varatt.h"
PG_FUNCTION_INFO_V1(gin_extract_trgm);
PG_FUNCTION_INFO_V1(gin_extract_value_trgm);
diff --git a/contrib/pg_trgm/trgm_gist.c b/contrib/pg_trgm/trgm_gist.c
index 3d74a1463a..ef5d8cca78 100644
--- a/contrib/pg_trgm/trgm_gist.c
+++ b/contrib/pg_trgm/trgm_gist.c
@@ -8,6 +8,7 @@
#include "fmgr.h"
#include "port/pg_bitutils.h"
#include "trgm.h"
+#include "varatt.h"
/* gist_trgm_ops opclass options */
typedef struct
diff --git a/contrib/pgcrypto/pgcrypto.c b/contrib/pgcrypto/pgcrypto.c
index 8f882f4c68..96447c5757 100644
--- a/contrib/pgcrypto/pgcrypto.c
+++ b/contrib/pgcrypto/pgcrypto.c
@@ -39,6 +39,7 @@
#include "px.h"
#include "utils/builtins.h"
#include "utils/uuid.h"
+#include "varatt.h"
PG_MODULE_MAGIC;
diff --git a/contrib/uuid-ossp/uuid-ossp.c b/contrib/uuid-ossp/uuid-ossp.c
index 1245af3abb..24b80f7113 100644
--- a/contrib/uuid-ossp/uuid-ossp.c
+++ b/contrib/uuid-ossp/uuid-ossp.c
@@ -19,6 +19,7 @@
#include "port/pg_bswap.h"
#include "utils/builtins.h"
#include "utils/uuid.h"
+#include "varatt.h"
/*
* It's possible that there's more than one uuid.h header file present.
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 7156ae9c47..367010461d 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -22,6 +22,7 @@
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#include "varatt.h"
/* GUC */
int default_toast_compression = TOAST_PGLZ_COMPRESSION;
diff --git a/src/backend/access/hash/hashfunc.c b/src/backend/access/hash/hashfunc.c
index f890f79ee1..fa70d16b29 100644
--- a/src/backend/access/hash/hashfunc.c
+++ b/src/backend/access/hash/hashfunc.c
@@ -32,6 +32,7 @@
#include "utils/builtins.h"
#include "utils/float.h"
#include "utils/pg_locale.h"
+#include "varatt.h"
/*
* Datatype-specific hash functions.
diff --git a/src/backend/access/spgist/spgtextproc.c b/src/backend/access/spgist/spgtextproc.c
index 199d921463..cf386ca7bb 100644
--- a/src/backend/access/spgist/spgtextproc.c
+++ b/src/backend/access/spgist/spgtextproc.c
@@ -46,6 +46,7 @@
#include "utils/datum.h"
#include "utils/pg_locale.h"
#include "utils/varlena.h"
+#include "varatt.h"
/*
diff --git a/src/backend/libpq/be-fsstubs.c b/src/backend/libpq/be-fsstubs.c
index c6e1e88dee..b202d40a13 100644
--- a/src/backend/libpq/be-fsstubs.c
+++ b/src/backend/libpq/be-fsstubs.c
@@ -53,6 +53,7 @@
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
+#include "varatt.h"
/* define this to enable debug logging */
/* #define FSDB 1 */
diff --git a/src/backend/tsearch/ts_parse.c b/src/backend/tsearch/ts_parse.c
index eff1669b12..34ea4fef0a 100644
--- a/src/backend/tsearch/ts_parse.c
+++ b/src/backend/tsearch/ts_parse.c
@@ -16,6 +16,7 @@
#include "tsearch/ts_cache.h"
#include "tsearch/ts_utils.h"
+#include "varatt.h"
#define IGNORE_LONGLEXEME 1
diff --git a/src/backend/tsearch/ts_typanalyze.c b/src/backend/tsearch/ts_typanalyze.c
index 187d9f16b1..b2134ca3b4 100644
--- a/src/backend/tsearch/ts_typanalyze.c
+++ b/src/backend/tsearch/ts_typanalyze.c
@@ -19,6 +19,7 @@
#include "common/hashfn.h"
#include "tsearch/ts_type.h"
#include "utils/builtins.h"
+#include "varatt.h"
/* A hash key for lexemes */
diff --git a/src/backend/utils/adt/ascii.c b/src/backend/utils/adt/ascii.c
index 54c36a7da2..0078ab9c80 100644
--- a/src/backend/utils/adt/ascii.c
+++ b/src/backend/utils/adt/ascii.c
@@ -14,6 +14,7 @@
#include "mb/pg_wchar.h"
#include "utils/ascii.h"
#include "utils/builtins.h"
+#include "varatt.h"
static void pg_to_ascii(unsigned char *src, unsigned char *src_end,
unsigned char *dest, int enc);
diff --git a/src/backend/utils/adt/char.c b/src/backend/utils/adt/char.c
index e50293bf14..dc9fe8d433 100644
--- a/src/backend/utils/adt/char.c
+++ b/src/backend/utils/adt/char.c
@@ -19,6 +19,7 @@
#include "libpq/pqformat.h"
#include "utils/builtins.h"
+#include "varatt.h"
#define ISOCTAL(c) (((c) >= '0') && ((c) <= '7'))
#define TOOCTAL(c) ((c) + '0')
diff --git a/src/backend/utils/adt/cryptohashfuncs.c b/src/backend/utils/adt/cryptohashfuncs.c
index 03d84ea217..21e9ba4ba5 100644
--- a/src/backend/utils/adt/cryptohashfuncs.c
+++ b/src/backend/utils/adt/cryptohashfuncs.c
@@ -17,6 +17,7 @@
#include "common/md5.h"
#include "common/sha2.h"
#include "utils/builtins.h"
+#include "varatt.h"
/*
diff --git a/src/backend/utils/adt/encode.c b/src/backend/utils/adt/encode.c
index f3bb5cca43..0473410c18 100644
--- a/src/backend/utils/adt/encode.c
+++ b/src/backend/utils/adt/encode.c
@@ -18,6 +18,7 @@
#include "mb/pg_wchar.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "varatt.h"
/*
diff --git a/src/backend/utils/adt/formatting.c b/src/backend/utils/adt/formatting.c
index 65746c48d2..0751e4146d 100644
--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -88,6 +88,7 @@
#include "utils/memutils.h"
#include "utils/numeric.h"
#include "utils/pg_locale.h"
+#include "varatt.h"
/* ----------
diff --git a/src/backend/utils/adt/geo_ops.c b/src/backend/utils/adt/geo_ops.c
index 86da025906..c34a8a1481 100644
--- a/src/backend/utils/adt/geo_ops.c
+++ b/src/backend/utils/adt/geo_ops.c
@@ -35,6 +35,7 @@
#include "utils/float.h"
#include "utils/fmgrprotos.h"
#include "utils/geo_decls.h"
+#include "varatt.h"
/*
* * Type constructors have this form:
diff --git a/src/backend/utils/adt/like.c b/src/backend/utils/adt/like.c
index 8e671b9fab..b3bfe5904c 100644
--- a/src/backend/utils/adt/like.c
+++ b/src/backend/utils/adt/like.c
@@ -24,6 +24,7 @@
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/pg_locale.h"
+#include "varatt.h"
#define LIKE_TRUE 1
diff --git a/src/backend/utils/adt/network_gist.c b/src/backend/utils/adt/network_gist.c
index 95d16dffe0..1f3b31fb90 100644
--- a/src/backend/utils/adt/network_gist.c
+++ b/src/backend/utils/adt/network_gist.c
@@ -51,6 +51,7 @@
#include "access/stratnum.h"
#include "utils/builtins.h"
#include "utils/inet.h"
+#include "varatt.h"
/*
* Operator strategy numbers used in the GiST inet_ops opclass
diff --git a/src/backend/utils/adt/network_spgist.c b/src/backend/utils/adt/network_spgist.c
index 3832156941..6ab3b40ee9 100644
--- a/src/backend/utils/adt/network_spgist.c
+++ b/src/backend/utils/adt/network_spgist.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "utils/builtins.h"
#include "utils/inet.h"
+#include "varatt.h"
static int inet_spg_node_number(const inet *val, int commonbits);
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index 018e8c9342..c6d127417c 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -21,6 +21,7 @@
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/memutils.h"
+#include "varatt.h"
static text *dotrim(const char *string, int stringlen,
diff --git a/src/backend/utils/adt/quote.c b/src/backend/utils/adt/quote.c
index 0a466740d9..8a405dc4fd 100644
--- a/src/backend/utils/adt/quote.c
+++ b/src/backend/utils/adt/quote.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "utils/builtins.h"
+#include "varatt.h"
/*
diff --git a/src/backend/utils/adt/rangetypes.c b/src/backend/utils/adt/rangetypes.c
index 2817b5e06e..99cb5f0310 100644
--- a/src/backend/utils/adt/rangetypes.c
+++ b/src/backend/utils/adt/rangetypes.c
@@ -42,6 +42,7 @@
#include "utils/lsyscache.h"
#include "utils/rangetypes.h"
#include "utils/timestamp.h"
+#include "varatt.h"
/* fn_extra cache entry for one of the range I/O functions */
diff --git a/src/backend/utils/adt/rangetypes_typanalyze.c b/src/backend/utils/adt/rangetypes_typanalyze.c
index f73c904b90..661f9c41e0 100644
--- a/src/backend/utils/adt/rangetypes_typanalyze.c
+++ b/src/backend/utils/adt/rangetypes_typanalyze.c
@@ -31,6 +31,7 @@
#include "utils/lsyscache.h"
#include "utils/rangetypes.h"
#include "utils/multirangetypes.h"
+#include "varatt.h"
static int float8_qsort_cmp(const void *a1, const void *a2, void *arg);
static int range_bound_qsort_cmp(const void *a1, const void *a2, void *arg);
diff --git a/src/backend/utils/adt/tsginidx.c b/src/backend/utils/adt/tsginidx.c
index cf23aeb5ea..d0a5a587e9 100644
--- a/src/backend/utils/adt/tsginidx.c
+++ b/src/backend/utils/adt/tsginidx.c
@@ -19,6 +19,7 @@
#include "tsearch/ts_type.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
+#include "varatt.h"
Datum
diff --git a/src/backend/utils/adt/tsquery.c b/src/backend/utils/adt/tsquery.c
index 1097294d55..885f793e86 100644
--- a/src/backend/utils/adt/tsquery.c
+++ b/src/backend/utils/adt/tsquery.c
@@ -23,6 +23,7 @@
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/pg_crc.h"
+#include "varatt.h"
/* FTS operator priorities, see ts_type.h */
const int tsearch_op_priority[OP_COUNT] =
diff --git a/src/backend/utils/adt/tsquery_cleanup.c b/src/backend/utils/adt/tsquery_cleanup.c
index 94030a75d5..5d5f675f90 100644
--- a/src/backend/utils/adt/tsquery_cleanup.c
+++ b/src/backend/utils/adt/tsquery_cleanup.c
@@ -17,6 +17,7 @@
#include "miscadmin.h"
#include "tsearch/ts_utils.h"
+#include "varatt.h"
typedef struct NODE
{
diff --git a/src/backend/utils/adt/tsquery_op.c b/src/backend/utils/adt/tsquery_op.c
index 8d815fa7a8..a041294757 100644
--- a/src/backend/utils/adt/tsquery_op.c
+++ b/src/backend/utils/adt/tsquery_op.c
@@ -17,6 +17,7 @@
#include "lib/qunique.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
+#include "varatt.h"
Datum
tsquery_numnode(PG_FUNCTION_ARGS)
diff --git a/src/backend/utils/adt/tsquery_util.c b/src/backend/utils/adt/tsquery_util.c
index 191b58e177..efa5370b8a 100644
--- a/src/backend/utils/adt/tsquery_util.c
+++ b/src/backend/utils/adt/tsquery_util.c
@@ -16,6 +16,7 @@
#include "miscadmin.h"
#include "tsearch/ts_utils.h"
+#include "varatt.h"
/*
* Build QTNode tree for a tsquery given in QueryItem array format.
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index 0b430d3c47..e498b704c1 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -20,6 +20,7 @@
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "varatt.h"
typedef struct
{
diff --git a/src/backend/utils/cache/attoptcache.c b/src/backend/utils/cache/attoptcache.c
index 9e252a0891..9f4db27c05 100644
--- a/src/backend/utils/cache/attoptcache.c
+++ b/src/backend/utils/cache/attoptcache.c
@@ -22,6 +22,7 @@
#include "utils/hsearch.h"
#include "utils/inval.h"
#include "utils/syscache.h"
+#include "varatt.h"
/* Hash table for information about each attribute's options */
diff --git a/src/backend/utils/cache/spccache.c b/src/backend/utils/cache/spccache.c
index 5609246c2f..acbd9a1f18 100644
--- a/src/backend/utils/cache/spccache.c
+++ b/src/backend/utils/cache/spccache.c
@@ -29,6 +29,7 @@
#include "utils/inval.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
+#include "varatt.h"
/* Hash table for information about each tablespace */
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 24f37e3ec9..b90e75c685 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -40,6 +40,7 @@
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/syscache.h"
+#include "varatt.h"
/*
* We maintain a simple linked list caching the fmgr lookup info for the
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index 0ce0090360..972afe3aff 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -18,8 +18,6 @@
#ifndef FMGR_H
#define FMGR_H
-#include "varatt.h"
-
/* We don't want to include primnodes.h here, so make some stub references */
typedef struct Node *fmNodePtr;
typedef struct Aggref *fmAggrefPtr;
diff --git a/src/pl/plperl/Util.xs b/src/pl/plperl/Util.xs
index bb4580ebfa..0ecb87fe33 100644
--- a/src/pl/plperl/Util.xs
+++ b/src/pl/plperl/Util.xs
@@ -16,6 +16,7 @@
#include "fmgr.h"
#include "utils/builtins.h"
#include "utils/bytea.h" /* for byteain & byteaout */
+#include "varatt.h"
/* perl stuff */
#define PG_NEED_PERL_XSUB_H
diff --git a/src/test/modules/spgist_name_ops/spgist_name_ops.c b/src/test/modules/spgist_name_ops/spgist_name_ops.c
index 89595fee93..b380111445 100644
--- a/src/test/modules/spgist_name_ops/spgist_name_ops.c
+++ b/src/test/modules/spgist_name_ops/spgist_name_ops.c
@@ -24,6 +24,7 @@
#include "access/spgist.h"
#include "catalog/pg_type.h"
#include "utils/datum.h"
+#include "varatt.h"
PG_MODULE_MAGIC;
diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c
index d8e513ae93..0874b5d595 100644
--- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c
+++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c
@@ -22,6 +22,7 @@
#include "access/xloginsert.h"
#include "fmgr.h"
#include "utils/pg_lsn.h"
+#include "varatt.h"
PG_MODULE_MAGIC;
diff --git a/src/test/modules/test_shm_mq/test.c b/src/test/modules/test_shm_mq/test.c
index 1d1c184d8c..eb79c4f26f 100644
--- a/src/test/modules/test_shm_mq/test.c
+++ b/src/test/modules/test_shm_mq/test.c
@@ -16,6 +16,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "varatt.h"
#include "test_shm_mq.h"
--
2.39.0
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 28.12.22 16:07, Tom Lane wrote:
I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included
so widely, I doubt it is buying very much in terms of reducing header
footprint. How bad is it to do #2?
See this incremental patch set.
Wow, 41 files requiring varatt.h is a lot fewer than I would have guessed.
I think that bears out my feeling that fmgr.h wasn't a great location:
I count 117 #includes of that, many of which are in .h files themselves
so that many more .c files would be required to read them.
(You did check that this passes cpluspluscheck/headerscheck, right?)
It seems like maybe there is some intermediate abstraction that a lot of
these places should be using that we haven't thought of yet.
Hmm. Perhaps, but I think I'm content with this version of the patch.
regards, tom lane
On 2022-12-30 Fr 11:50, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 28.12.22 16:07, Tom Lane wrote:
I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included
so widely, I doubt it is buying very much in terms of reducing header
footprint. How bad is it to do #2?See this incremental patch set.
Wow, 41 files requiring varatt.h is a lot fewer than I would have guessed.
I think that bears out my feeling that fmgr.h wasn't a great location:
I count 117 #includes of that, many of which are in .h files themselves
so that many more .c files would be required to read them.(You did check that this passes cpluspluscheck/headerscheck, right?)
It seems like maybe there is some intermediate abstraction that a lot of
these places should be using that we haven't thought of yet.Hmm. Perhaps, but I think I'm content with this version of the patch.
Looked good to me too.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On 30.12.22 17:50, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 28.12.22 16:07, Tom Lane wrote:
I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included
so widely, I doubt it is buying very much in terms of reducing header
footprint. How bad is it to do #2?See this incremental patch set.
Wow, 41 files requiring varatt.h is a lot fewer than I would have guessed.
I think that bears out my feeling that fmgr.h wasn't a great location:
I count 117 #includes of that, many of which are in .h files themselves
so that many more .c files would be required to read them.
committed
On Tue, Jan 10, 2023 at 06:07:49AM +0100, Peter Eisentraut wrote:
On 30.12.22 17:50, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 28.12.22 16:07, Tom Lane wrote:
I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included
so widely, I doubt it is buying very much in terms of reducing header
footprint. How bad is it to do #2?See this incremental patch set.
Wow, 41 files requiring varatt.h is a lot fewer than I would have guessed.
I think that bears out my feeling that fmgr.h wasn't a great location:
I count 117 #includes of that, many of which are in .h files themselves
so that many more .c files would be required to read them.committed
SET_VARSIZE alone appears in 74 pgxn distributions, so I predict extension
breakage en masse. I would revert this.
On 10.01.23 08:39, Noah Misch wrote:
On Tue, Jan 10, 2023 at 06:07:49AM +0100, Peter Eisentraut wrote:
On 30.12.22 17:50, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 28.12.22 16:07, Tom Lane wrote:
I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included
so widely, I doubt it is buying very much in terms of reducing header
footprint. How bad is it to do #2?See this incremental patch set.
Wow, 41 files requiring varatt.h is a lot fewer than I would have guessed.
I think that bears out my feeling that fmgr.h wasn't a great location:
I count 117 #includes of that, many of which are in .h files themselves
so that many more .c files would be required to read them.committed
SET_VARSIZE alone appears in 74 pgxn distributions, so I predict extension
breakage en masse. I would revert this.
Well, that was sort of my thinking, but people seemed to like this. I'm
happy to consider alternatives.
On Tue, Jan 10, 2023 at 3:48 AM Peter Eisentraut
<peter.eisentraut@enterprisedb.com> wrote:
Wow, 41 files requiring varatt.h is a lot fewer than I would have guessed.
I think that bears out my feeling that fmgr.h wasn't a great location:
I count 117 #includes of that, many of which are in .h files themselves
so that many more .c files would be required to read them.committed
SET_VARSIZE alone appears in 74 pgxn distributions, so I predict extension
breakage en masse. I would revert this.Well, that was sort of my thinking, but people seemed to like this. I'm
happy to consider alternatives.
I don't think that the number of extensions that get broken is really
the right metric. It's not fantastic to break large numbers of
extensions, of course, but if the solution is merely to add an #if
PG_VERSION_NUM >= whatever #include "newstuff" #endif then I don't
think it's really an issue. If an extension doesn't have an author who
can do at least that much updating when a new PG release comes out,
then it's basically unmaintained, and I just don't feel that bad about
breaking unmaintained extensions now and then, even annually.
Of course, if we go and remove something that's very widely used and
for which there's no simple workaround, that sucks. Say, removing
LWLocks entirely. But we don't usually do that sort of thing unless
there's a good reason and significant benefits.
I don't think it would be very nice to do something like this in a
minor release. But in a new major release, I think it's fine. I've
been on the hook to maintain extensions in the face of these kinds of
changes at various times over the years, and it's never taken me much
time.
--
Robert Haas
EDB: http://www.enterprisedb.com
Robert Haas <robertmhaas@gmail.com> writes:
On Tue, Jan 10, 2023 at 3:48 AM Peter Eisentraut
<peter.eisentraut@enterprisedb.com> wrote:SET_VARSIZE alone appears in 74 pgxn distributions, so I predict extension
breakage en masse. I would revert this.
Well, that was sort of my thinking, but people seemed to like this. I'm
happy to consider alternatives.
I don't think it would be very nice to do something like this in a
minor release. But in a new major release, I think it's fine. I've
been on the hook to maintain extensions in the face of these kinds of
changes at various times over the years, and it's never taken me much
time.
Yeah, that was my thinking. We could never do any header refactoring
at all if the standard is "will some extension author need to add a #if".
In practice, we make bigger adjustments than this all the time,
both in header layout and in individual function APIs.
Now, there is a fair question whether splitting this code out of
postgres.h is worth any trouble at all. TBH my initial reaction
had been "no". But once we found that only 40-ish backend files
need to read this new header, I became a "yes" vote because it
seems clear that there will be a total-compilation-time benefit.
regards, tom lane
On Tue, Jan 10, 2023 at 9:46 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Now, there is a fair question whether splitting this code out of
postgres.h is worth any trouble at all. TBH my initial reaction
had been "no". But once we found that only 40-ish backend files
need to read this new header, I became a "yes" vote because it
seems clear that there will be a total-compilation-time benefit.
I wasn't totally about this, either, but I think on balance it's
probably a smart thing to do. I always found it kind of weird that we
put that stuff in postgres.h. It seems to privilege the TOAST
mechanism to an undue degree; what's the argument, for example, that
TOAST macros are more generally relevant than CHECK_FOR_INTERRUPTS()
or LWLockAcquire or HeapTuple? It felt to me like we'd just decided
that one subsystem gets to go into the main header file and everybody
else just had to have their own headers.
One thing that's particularly awkward about that is that if you want
to write some front-end code that knows about how varlenas are stored
on disk, it was very awkward with the old structure. You're not
supposed to include "postgres.h" into frontend code, but if the stuff
you need is defined there then what else can you do? So I generally
think that anything that's in postgres.h should have a strong claim to
be not only widely-needed in the backend, but also never needed at all
in any other executable.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Tue, Jan 10, 2023 at 12:00:46PM -0500, Robert Haas wrote:
On Tue, Jan 10, 2023 at 9:46 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Now, there is a fair question whether splitting this code out of
postgres.h is worth any trouble at all. TBH my initial reaction
had been "no". But once we found that only 40-ish backend files
need to read this new header, I became a "yes" vote because it
seems clear that there will be a total-compilation-time benefit.
A time claim with no benchmarks is a red flag. I've chosen to run one:
export CCACHE_DISABLE=1
change=d952373a987bad331c0e499463159dd142ced1ef
for commit in $change $change^; do
echo === git checkout $commit
git checkout $commit
for n in `seq 1 200`; do make -j20 clean; env time make -j20; done
done
Results:
commit median mean count
d952373a987bad331c0e499463159dd142ced1ef 49.35 49.37 200
d952373a987bad331c0e499463159dd142ced1ef^ 49.33 49.36 200
That is to say, the patch made the build a bit slower, not faster. That's
with GCC 4.8.5 (RHEL 7). I likely should have interleaved the run types, but
in any case the speed win didn't show up.
I wasn't totally about this, either, but I think on balance it's
probably a smart thing to do. I always found it kind of weird that we
put that stuff in postgres.h. It seems to privilege the TOAST
mechanism to an undue degree; what's the argument, for example, that
TOAST macros are more generally relevant than CHECK_FOR_INTERRUPTS()
or LWLockAcquire or HeapTuple? It felt to me like we'd just decided
that one subsystem gets to go into the main header file and everybody
else just had to have their own headers.One thing that's particularly awkward about that is that if you want
to write some front-end code that knows about how varlenas are stored
on disk, it was very awkward with the old structure. You're not
supposed to include "postgres.h" into frontend code, but if the stuff
you need is defined there then what else can you do? So I generally
think that anything that's in postgres.h should have a strong claim to
be not only widely-needed in the backend, but also never needed at all
in any other executable.
If the patch had just made postgres.h include varatt.h, like it does elog.h,
I'd consider that change a nonnegative. Grouping things is nice, even if it
makes compilation a bit slower. That also covers your frontend use case. How
about that?
I agree fixing any one extension is easy enough. Thinking back to the
htup_details.h refactor, I found the aggregate pain unreasonable relative to
alleged benefits, even though each individual extension wasn't too bad.
SET_VARSIZE is used more (74 pgxn distributions) than htup_details.h (62).
On Wed, Jan 11, 2023 at 1:14 AM Noah Misch <noah@leadboat.com> wrote:
If the patch had just made postgres.h include varatt.h, like it does elog.h,
I'd consider that change a nonnegative. Grouping things is nice, even if it
makes compilation a bit slower. That also covers your frontend use case. How
about that?
I'm not direly opposed to that, but I'm also unconvinced that having
the varatt.h stuff is important enough relative to other things to
justify giving it a privileged place forever.
I agree fixing any one extension is easy enough. Thinking back to the
htup_details.h refactor, I found the aggregate pain unreasonable relative to
alleged benefits, even though each individual extension wasn't too bad.
SET_VARSIZE is used more (74 pgxn distributions) than htup_details.h (62).
What annoyed me about that refactoring is that, in most cases where I
had been including htup.h, I had to change it to include
htup_details.h. In the main source tree, too, we've got 31 places that
include access/htup.h and 241 that include access/htup_details.h. It's
hard to argue that it was worth splitting the file given those
numbers, and in fact I think that change was a mistake. But that
doesn't mean every such change is a mistake.
--
Robert Haas
EDB: http://www.enterprisedb.com
On 10.01.23 09:48, Peter Eisentraut wrote:
On 10.01.23 08:39, Noah Misch wrote:
On Tue, Jan 10, 2023 at 06:07:49AM +0100, Peter Eisentraut wrote:
On 30.12.22 17:50, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 28.12.22 16:07, Tom Lane wrote:
I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is
included
so widely, I doubt it is buying very much in terms of reducing header
footprint. How bad is it to do #2?See this incremental patch set.
Wow, 41 files requiring varatt.h is a lot fewer than I would have
guessed.
I think that bears out my feeling that fmgr.h wasn't a great location:
I count 117 #includes of that, many of which are in .h files themselves
so that many more .c files would be required to read them.committed
SET_VARSIZE alone appears in 74 pgxn distributions, so I predict
extension
breakage en masse. I would revert this.Well, that was sort of my thinking, but people seemed to like this. I'm
happy to consider alternatives.
Given the subsequent discussion, I'll keep it as is for now but consider
it a semi-open item. It's easy to change.