[Fwd: Re: [PATCHES] 64-bit CommandIds]
Hi,
what's your opinion on this?
I saw response only from Alvaro on the -patches list.
Thanks in advance,
Zoltán Böszörményi
-------- Eredeti üzenet --------
Tárgy: Re: [PATCHES] 64-bit CommandIds
Dátum: Tue, 04 Mar 2008 21:52:25 +0100
Feladó: Zoltan Boszormenyi <zb@cybertec.at>
Címzett: pgsql-patches <pgsql-patches@postgresql.org>
CC: Alvaro Herrera <alvherre@commandprompt.com>, Hans-Juergen Schoenig
<hs@cybertec.at>
Hivatkozások: <47CD8665.7070903@cybertec.at>
<20080304174110.GK4755@alvh.no-ip.org>
Alvaro Herrera írta:
Zoltan Boszormenyi wrote:
attached is our patch against HEAD which enables extending CommandIds
to 64-bit. This is for enabling long transactions that really do that much
non-read-only work in one transaction.I think you should add a pg_control field and corresponding check, to
avoid a 64bit-Cid postmaster to start on a 32bit-Cid data area and vice
versa.
I added the check but I needed to add it BEFORE checking for
toast_max_chunk_size otherwise it complained about this more
cryptic problem. I think it's cleaner to report this failure to know
why toast_max_chunk_size != TOAST_MAX_CHUNK_SIZE.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/
Attachments:
pg84-64-bit-cmdid-v2.patchtext/x-patch; charset=iso-8859-1; name=pg84-64-bit-cmdid-v2.patchDownload
diff -dcrpN pgsql.orig/configure pgsql-cid64/configure
*** pgsql.orig/configure 2008-03-02 13:44:42.000000000 +0100
--- pgsql-cid64/configure 2008-03-04 16:53:46.000000000 +0100
*************** if test -n "$ac_init_help"; then
*** 1349,1354 ****
--- 1349,1355 ----
Optional Features:
--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
--enable-FEATURE[=ARG] include FEATURE [ARG=yes]
+ --enable-huge-commandid enable 64-bit CommandId support
--enable-integer-datetimes enable 64-bit integer date/time support
--enable-nls[=LANGUAGES] enable Native Language Support
--disable-shared do not build shared libraries
*************** fi
*** 2175,2180 ****
--- 2176,2219 ----
#
+ # 64-bit CommandId
+ #
+ echo "$as_me:$LINENO: checking whether to build with 64-bit CommandId support" >&5
+ echo $ECHO_N "checking whether to build with 64-bit CommandId support... $ECHO_C" >&6
+
+ pgac_args="$pgac_args enable_huge_commandid"
+
+ # Check whether --enable-huge-commandid or --disable-huge-commandid was given.
+ if test "${enable_huge_commandid+set}" = set; then
+ enableval="$enable_huge_commandid"
+
+ case $enableval in
+ yes)
+
+ cat >>confdefs.h <<\_ACEOF
+ #define USE_64BIT_COMMANDID 1
+ _ACEOF
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ { { echo "$as_me:$LINENO: error: no argument expected for --enable-huge-commandid option" >&5
+ echo "$as_me: error: no argument expected for --enable-huge-commandid option" >&2;}
+ { (exit 1); exit 1; }; }
+ ;;
+ esac
+
+ else
+ enable_huge_commandid=no
+
+ fi;
+
+ echo "$as_me:$LINENO: result: $enable_huge_commandid" >&5
+ echo "${ECHO_T}$enable_huge_commandid" >&6
+
+ #
# 64-bit integer date/time storage (--enable-integer-datetimes)
#
{ echo "$as_me:$LINENO: checking whether to build with 64-bit integer date/time support" >&5
diff -dcrpN pgsql.orig/configure.in pgsql-cid64/configure.in
*** pgsql.orig/configure.in 2008-03-02 13:44:43.000000000 +0100
--- pgsql-cid64/configure.in 2008-03-04 16:53:46.000000000 +0100
*************** PGAC_ARG_REQ(with, libs, [ --with-
*** 128,133 ****
--- 128,142 ----
#
+ # 64-bit CommandId
+ #
+ AC_MSG_CHECKING([whether to build with 64-bit CommandId support])
+ PGAC_ARG_BOOL(enable, huge-commandid, no, [ --enable-huge-commandid enable 64-bit CommandId support],
+ [AC_DEFINE([USE_64BIT_COMMANDID], 1,
+ [Define to 1 if you want 64-bit CommandId support. (--enable-huge-commandid)])])
+ AC_MSG_RESULT([$enable_huge_commandid])
+
+ #
# 64-bit integer date/time storage (--enable-integer-datetimes)
#
AC_MSG_CHECKING([whether to build with 64-bit integer date/time support])
diff -dcrpN pgsql.orig/doc/src/sgml/installation.sgml pgsql-cid64/doc/src/sgml/installation.sgml
*** pgsql.orig/doc/src/sgml/installation.sgml 2008-02-18 13:49:58.000000000 +0100
--- pgsql-cid64/doc/src/sgml/installation.sgml 2008-03-04 17:16:14.000000000 +0100
*************** su - postgres
*** 1011,1016 ****
--- 1011,1027 ----
</varlistentry>
<varlistentry>
+ <term><option>--enable-huge-commandid</option></term>
+ <listitem>
+ <para>
+ Use 64-bit CommandIds if you are planning to run transactions
+ consisting of more than 4 billion commands. This is off by default
+ to save disk space.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>--enable-integer-datetimes</option></term>
<listitem>
<para>
diff -dcrpN pgsql.orig/src/backend/access/transam/xact.c pgsql-cid64/src/backend/access/transam/xact.c
*** pgsql.orig/src/backend/access/transam/xact.c 2008-01-15 19:56:59.000000000 +0100
--- pgsql-cid64/src/backend/access/transam/xact.c 2008-03-04 16:57:54.000000000 +0100
*************** CommandCounterIncrement(void)
*** 592,598 ****
--- 592,602 ----
currentCommandId -= 1;
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ #ifdef USE_64BIT_COMMANDID
+ errmsg("cannot have more than 2^64-1 commands in a transaction")));
+ #else
errmsg("cannot have more than 2^32-1 commands in a transaction")));
+ #endif
}
currentCommandIdUsed = false;
diff -dcrpN pgsql.orig/src/backend/access/transam/xlog.c pgsql-cid64/src/backend/access/transam/xlog.c
*** pgsql.orig/src/backend/access/transam/xlog.c 2008-02-18 13:50:00.000000000 +0100
--- pgsql-cid64/src/backend/access/transam/xlog.c 2008-03-04 21:39:30.000000000 +0100
*************** WriteControlFile(void)
*** 3794,3799 ****
--- 3794,3805 ----
ControlFile->enableIntTimes = FALSE;
#endif
+ #ifdef USE_64BIT_COMMANDID
+ ControlFile->enable64bitCommandId = TRUE;
+ #else
+ ControlFile->enable64bitCommandId = FALSE;
+ #endif
+
ControlFile->localeBuflen = LOCALE_NAME_BUFLEN;
localeptr = setlocale(LC_COLLATE, NULL);
if (!localeptr)
*************** ReadControlFile(void)
*** 3989,3994 ****
--- 3995,4017 ----
" but the server was compiled with INDEX_MAX_KEYS %d.",
ControlFile->indexMaxKeys, INDEX_MAX_KEYS),
errhint("It looks like you need to recompile or initdb.")));
+
+ #ifdef USE_64BIT_COMMANDID
+ if (ControlFile->enable64bitCommandId != TRUE)
+ ereport(FATAL,
+ (errmsg("database files are incompatible with server"),
+ errdetail("The database cluster was initialized without USE_64BIT_COMMANDID"
+ " but the server was compiled with USE_64BIT_COMMANDID."),
+ errhint("It looks like you need to recompile or initdb.")));
+ #else
+ if (ControlFile->enable64bitCommandId != FALSE)
+ ereport(FATAL,
+ (errmsg("database files are incompatible with server"),
+ errdetail("The database cluster was initialized with USE_64BIT_COMMANDID"
+ " but the server was compiled without USE_64BIT_COMMANDID."),
+ errhint("It looks like you need to recompile or initdb.")));
+ #endif
+
if (ControlFile->toast_max_chunk_size != TOAST_MAX_CHUNK_SIZE)
ereport(FATAL,
(errmsg("database files are incompatible with server"),
diff -dcrpN pgsql.orig/src/backend/bootstrap/bootstrap.c pgsql-cid64/src/backend/bootstrap/bootstrap.c
*** pgsql.orig/src/backend/bootstrap/bootstrap.c 2008-02-18 13:50:00.000000000 +0100
--- pgsql-cid64/src/backend/bootstrap/bootstrap.c 2008-03-04 16:53:46.000000000 +0100
*************** static const struct typinfo TypInfo[] =
*** 139,145 ****
--- 139,149 ----
F_TIDIN, F_TIDOUT},
{"xid", XIDOID, 0, 4, true, 'i', 'p',
F_XIDIN, F_XIDOUT},
+ #ifdef USE_64BIT_COMMANDID
+ {"cid", CIDOID, 0, 8, false, 'd', 'p',
+ #else
{"cid", CIDOID, 0, 4, true, 'i', 'p',
+ #endif
F_CIDIN, F_CIDOUT},
{"int2vector", INT2VECTOROID, INT2OID, -1, false, 'i', 'p',
F_INT2VECTORIN, F_INT2VECTOROUT},
diff -dcrpN pgsql.orig/src/backend/catalog/genbki.sh pgsql-cid64/src/backend/catalog/genbki.sh
*** pgsql.orig/src/backend/catalog/genbki.sh 2008-01-01 20:45:48.000000000 +0100
--- pgsql-cid64/src/backend/catalog/genbki.sh 2008-03-04 16:53:46.000000000 +0100
*************** for dir in $INCLUDE_DIRS; do
*** 114,119 ****
--- 114,136 ----
fi
done
+ # Deduce CIDSTORAGELEN from pg_config.h / USE_64BIT_COMMANDID
+ for dir in $INCLUDE_DIRS; do
+ if [ -f "$dir/pg_config.h" ]; then
+ HUGECID=`grep '^#define[ ]*USE_64BIT_COMMANDID' $dir/pg_config.h | $AWK '{ print $3 }'`
+ break
+ fi
+ done
+ if [ "$HUGECID" == "1" ]; then
+ CIDSTORAGELEN="8"
+ CIDSTORAGEALIGN="d"
+ CIDPASSBYVAL="f"
+ else
+ CIDSTORAGELEN="4"
+ CIDSTORAGEALIGN="i"
+ CIDPASSBYVAL="t"
+ fi
+
# Get BOOTSTRAP_SUPERUSERID from catalog/pg_authid.h
for dir in $INCLUDE_DIRS; do
if [ -f "$dir/catalog/pg_authid.h" ]; then
*************** sed -e "s/;[ ]*$//g" \
*** 164,169 ****
--- 181,189 ----
-e "s/(TransactionId/(xid/g" \
-e "s/PGUID/$BOOTSTRAP_SUPERUSERID/g" \
-e "s/NAMEDATALEN/$NAMEDATALEN/g" \
+ -e "s/CIDSTORAGELEN/$CIDSTORAGELEN/g" \
+ -e "s/CIDSTORAGEALIGN/$CIDSTORAGEALIGN/g" \
+ -e "s/CIDPASSBYVAL/$CIDPASSBYVAL/g" \
-e "s/PGNSP/$PG_CATALOG_NAMESPACE/g" \
| $AWK '
# ----------------
diff -dcrpN pgsql.orig/src/backend/catalog/heap.c pgsql-cid64/src/backend/catalog/heap.c
*** pgsql.orig/src/backend/catalog/heap.c 2008-01-01 20:45:48.000000000 +0100
--- pgsql-cid64/src/backend/catalog/heap.c 2008-03-04 16:53:46.000000000 +0100
*************** static FormData_pg_attribute a3 = {
*** 117,123 ****
--- 117,127 ----
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
+ #ifdef USE_64BIT_COMMANDID
+ false, 'p', 'd', true, false, false, true, 0
+ #else
true, 'p', 'i', true, false, false, true, 0
+ #endif
};
static FormData_pg_attribute a5 = {
*************** static FormData_pg_attribute a5 = {
*** 129,135 ****
--- 133,143 ----
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
+ #ifdef USE_64BIT_COMMANDID
+ false, 'p', 'd', true, false, false, true, 0
+ #else
true, 'p', 'i', true, false, false, true, 0
+ #endif
};
/*
diff -dcrpN pgsql.orig/src/backend/catalog/Makefile pgsql-cid64/src/backend/catalog/Makefile
*** pgsql.orig/src/backend/catalog/Makefile 2008-03-02 13:44:44.000000000 +0100
--- pgsql-cid64/src/backend/catalog/Makefile 2008-03-04 16:55:51.000000000 +0100
*************** postgres.description: postgres.bki ;
*** 46,52 ****
postgres.shdescription: postgres.bki ;
! postgres.bki: genbki.sh $(POSTGRES_BKI_SRCS) $(top_builddir)/src/include/pg_config_manual.h
AWK='$(AWK)' $(SHELL) $< $(pg_includes) --set-version=$(VERSION) -o postgres $(POSTGRES_BKI_SRCS)
.PHONY: install-data
--- 46,52 ----
postgres.shdescription: postgres.bki ;
! postgres.bki: genbki.sh $(POSTGRES_BKI_SRCS) $(top_builddir)/src/include/pg_config_manual.h $(top_builddir)/src/include/pg_config.h
AWK='$(AWK)' $(SHELL) $< $(pg_includes) --set-version=$(VERSION) -o postgres $(POSTGRES_BKI_SRCS)
.PHONY: install-data
diff -dcrpN pgsql.orig/src/include/catalog/pg_attribute.h pgsql-cid64/src/include/catalog/pg_attribute.h
*** pgsql.orig/src/include/catalog/pg_attribute.h 2008-01-01 20:45:56.000000000 +0100
--- pgsql-cid64/src/include/catalog/pg_attribute.h 2008-03-04 16:53:46.000000000 +0100
*************** DATA(insert ( 1247 typdefault 25 -1 -1
*** 278,286 ****
DATA(insert ( 1247 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0));
DATA(insert ( 1247 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1247 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1247 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1247 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1247 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1247 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0));
/* ----------------
--- 278,286 ----
DATA(insert ( 1247 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0));
DATA(insert ( 1247 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1247 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1247 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0));
DATA(insert ( 1247 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1247 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0));
DATA(insert ( 1247 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0));
/* ----------------
*************** DATA(insert ( 1255 proacl 1034 -1 -1
*** 334,342 ****
DATA(insert ( 1255 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0));
DATA(insert ( 1255 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1255 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1255 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1255 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1255 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1255 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0));
/* ----------------
--- 334,342 ----
DATA(insert ( 1255 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0));
DATA(insert ( 1255 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1255 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1255 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0));
DATA(insert ( 1255 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1255 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0));
DATA(insert ( 1255 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0));
/* ----------------
*************** DATA(insert ( 1249 attinhcount 23 -1 4
*** 382,390 ****
DATA(insert ( 1249 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0));
/* no OIDs in pg_attribute */
DATA(insert ( 1249 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1249 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1249 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1249 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1249 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0));
/* ----------------
--- 382,390 ----
DATA(insert ( 1249 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0));
/* no OIDs in pg_attribute */
DATA(insert ( 1249 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1249 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0));
DATA(insert ( 1249 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1249 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0));
DATA(insert ( 1249 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0));
/* ----------------
*************** DATA(insert ( 1259 reloptions 1009 -1
*** 450,458 ****
DATA(insert ( 1259 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0));
DATA(insert ( 1259 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1259 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1259 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1259 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1259 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1259 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0));
/* ----------------
--- 450,458 ----
DATA(insert ( 1259 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0));
DATA(insert ( 1259 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0));
DATA(insert ( 1259 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1259 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0));
DATA(insert ( 1259 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0));
! DATA(insert ( 1259 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0));
DATA(insert ( 1259 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0));
/* ----------------
diff -dcrpN pgsql.orig/src/include/catalog/pg_control.h pgsql-cid64/src/include/catalog/pg_control.h
*** pgsql.orig/src/include/catalog/pg_control.h 2008-02-18 13:50:12.000000000 +0100
--- pgsql-cid64/src/include/catalog/pg_control.h 2008-03-04 21:14:04.000000000 +0100
*************** typedef struct ControlFileData
*** 145,150 ****
--- 145,152 ----
char lc_collate[LOCALE_NAME_BUFLEN];
char lc_ctype[LOCALE_NAME_BUFLEN];
+ uint32 enable64bitCommandId;
+
/* CRC of all above ... MUST BE LAST! */
pg_crc32 crc;
} ControlFileData;
diff -dcrpN pgsql.orig/src/include/catalog/pg_type.h pgsql-cid64/src/include/catalog/pg_type.h
*** pgsql.orig/src/include/catalog/pg_type.h 2008-01-01 20:45:57.000000000 +0100
--- pgsql-cid64/src/include/catalog/pg_type.h 2008-03-04 16:59:27.000000000 +0100
*************** DATA(insert OID = 28 ( xid PGNSP PGU
*** 314,320 ****
DESCR("transaction id");
#define XIDOID 28
! DATA(insert OID = 29 ( cid PGNSP PGUID 4 t b t \054 0 0 1012 cidin cidout cidrecv cidsend - - - i p f 0 -1 0 _null_ _null_ ));
DESCR("command identifier type, sequence in transaction id");
#define CIDOID 29
--- 314,320 ----
DESCR("transaction id");
#define XIDOID 28
! DATA(insert OID = 29 ( cid PGNSP PGUID CIDSTORAGELEN CIDPASSBYVAL b t \054 0 0 1012 cidin cidout cidrecv cidsend - - - CIDSTORAGEALIGN p f 0 -1 0 _null_ _null_ ));
DESCR("command identifier type, sequence in transaction id");
#define CIDOID 29
diff -dcrpN pgsql.orig/src/include/c.h pgsql-cid64/src/include/c.h
*** pgsql.orig/src/include/c.h 2008-03-02 13:44:45.000000000 +0100
--- pgsql-cid64/src/include/c.h 2008-03-04 21:05:23.000000000 +0100
*************** typedef TransactionId MultiXactId;
*** 382,388 ****
--- 382,392 ----
typedef uint32 MultiXactOffset;
+ #ifdef USE_64BIT_COMMANDID
+ typedef uint64 CommandId;
+ #else
typedef uint32 CommandId;
+ #endif
#define FirstCommandId ((CommandId) 0)
diff -dcrpN pgsql.orig/src/include/pg_config.h.in pgsql-cid64/src/include/pg_config.h.in
*** pgsql.orig/src/include/pg_config.h.in 2008-02-18 13:50:12.000000000 +0100
--- pgsql-cid64/src/include/pg_config.h.in 2008-03-04 16:53:46.000000000 +0100
***************
*** 656,661 ****
--- 656,665 ----
*/
#undef UINT64_FORMAT
+ /* Define to 1 if you want 64-bit CommandId support. (--enable-huge-commandid)
+ */
+ #undef USE_64BIT_COMMANDID
+
/* Define to 1 to build with assertion checks. (--enable-cassert) */
#undef USE_ASSERT_CHECKING
diff -dcrpN pgsql.orig/src/include/postgres.h pgsql-cid64/src/include/postgres.h
*** pgsql.orig/src/include/postgres.h 2008-01-01 20:45:56.000000000 +0100
--- pgsql-cid64/src/include/postgres.h 2008-03-04 16:53:46.000000000 +0100
*************** typedef Datum *DatumPtr;
*** 462,475 ****
--- 462,483 ----
* Returns command identifier value of a datum.
*/
+ #ifdef USE_64BIT_COMMANDID
+ #define DatumGetCommandId(X) ((CommandId) DatumGetInt64(X))
+ #else
#define DatumGetCommandId(X) ((CommandId) GET_4_BYTES(X))
+ #endif
/*
* CommandIdGetDatum
* Returns datum representation for a command identifier.
*/
+ #ifdef USE_64BIT_COMMANDID
+ #define CommandIdGetDatum(X) ((Datum) Int64GetDatum(X))
+ #else
#define CommandIdGetDatum(X) ((Datum) SET_4_BYTES(X))
+ #endif
/*
* DatumGetPointer
"Zoltan Boszormenyi" <zb@cybertec.at> writes:
Hi,
what's your opinion on this?
I saw response only from Alvaro on the -patches list.
I don't understand. The patch only affects configuration and SQL data type
code. It doesn't actually store the 64-bit commandid anywhere which would be
the actual hard part.
Do "phantom" command ids mean this all just works magically? Ie, the limit of
2^32 <cmin,cmax> pairs is still there but as long as you don't have to store
more than that many you get to have 2^64 raw ephemeral commandids?
--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's Slony Replication support!
Gregory Stark wrote:
I don't understand. The patch only affects configuration and SQL data type
code. It doesn't actually store the 64-bit commandid anywhere which would be
the actual hard part.
Sure it does, this is the significant part of the patch:
*** pgsql.orig/src/include/c.h 2008-03-02 13:44:45.000000000 +0100
--- pgsql-cid64/src/include/c.h 2008-03-04 21:05:23.000000000 +0100
*************** typedef TransactionId MultiXactId;
*** 382,388 ****
--- 382,392 ----typedef uint32 MultiXactOffset;
+ #ifdef USE_64BIT_COMMANDID
+ typedef uint64 CommandId;
+ #else
typedef uint32 CommandId;
+ #endif#define FirstCommandId ((CommandId) 0)
CommandId type is used in htup.h and elsewhere, which changes the
on-disk format.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
On Mar 10, 2008, at 12:06 PM, Heikki Linnakangas wrote:
Gregory Stark wrote:
I don't understand. The patch only affects configuration and SQL
data type
code. It doesn't actually store the 64-bit commandid anywhere
which would be
the actual hard part.Sure it does, this is the significant part of the patch:
*** pgsql.orig/src/include/c.h 2008-03-02 13:44:45.000000000 +0100 --- pgsql-cid64/src/include/c.h 2008-03-04 21:05:23.000000000 +0100 *************** typedef TransactionId MultiXactId; *** 382,388 **** --- 382,392 ----typedef uint32 MultiXactOffset;
+ #ifdef USE_64BIT_COMMANDID + typedef uint64 CommandId; + #else typedef uint32 CommandId; + #endif#define FirstCommandId ((CommandId) 0)
CommandId type is used in htup.h and elsewhere, which changes the
on-disk format.
If we're going to make this a ./configure option, ISTM we should do
the same with XID size as well. I know there are high-velocity
databases that could use that.
--
Decibel!, aka Jim C. Nasby, Database Architect decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828
Attachments:
"Decibel!" <decibel@decibel.org> writes:
If we're going to make this a ./configure option, ISTM we should do the same
with XID size as well. I know there are high-velocity databases that could use
that.
Keep in mind we just changed things so that read-only transactions don't
consume xids. That means you would have to be actually modifying 2-billion
records before wrap-around becomes an issue.
If you're modifying 2-billion records that quickly presumably you're going to
have other pressing reasons to run vacuum aside from xid freezing...
Also, consider that you're suggesting increasing the per-tuple overhead from
24 bytes to, if my arithmetic is right, 40 bytes.
So really you would need, say, a system with enough i/o bandwidth to handle
2-billion updates or inserts per day and with enough spare i/o bandwidth that
another 16-bytes on every one of those updates is ok, but without the ability
to run vacuum.
Also, we still have hope that the visibility map info will make running vacuum
even less of an imposition.
All that said I don't really see much reason not to make it an option. I just
don't think anyone really needs it. In 5-10 years though...
--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's 24x7 Postgres support!
Gregory Stark <stark@enterprisedb.com> writes:
All that said I don't really see much reason not to make it an option. I just
don't think anyone really needs it. In 5-10 years though...
The manpower we'd have to invest in making it work and keeping it
working would be enough reason ...
regards, tom lane
"Decibel!" <decibel@decibel.org> writes:
If we're going to make this a ./configure option, ISTM we should
do the same
with XID size as well. I know there are high-velocity databases
that could use
that.Keep in mind we just changed things so that read-only transactions
don't
consume xids. That means you would have to be actually modifying 2-
billion
records before wrap-around becomes an issue.If you're modifying 2-billion records that quickly presumably
you're going to
have other pressing reasons to run vacuum aside from xid freezing...Also, consider that you're suggesting increasing the per-tuple
overhead from
24 bytes to, if my arithmetic is right, 40 bytes.So really you would need, say, a system with enough i/o bandwidth
to handle
2-billion updates or inserts per day and with enough spare i/o
bandwidth that
another 16-bytes on every one of those updates is ok, but without
the ability
to run vacuum.Also, we still have hope that the visibility map info will make
running vacuum
even less of an imposition.All that said I don't really see much reason not to make it an
option. I just
don't think anyone really needs it. In 5-10 years though...
Doing this for XIDs is pretty useless this days.
It is only targeted for command ids which are consumed heavily by
stored procedure languages.
It happens once on a while that a complex business logic procedure
runs out of command ids inside a transaction.
the idea is to give users a chance to avoid that.
touching XIDs does not make sense to me at all.
many thanks,
hans
--
Cybertec Schönig & Schönig GmbH
PostgreSQL Solutions and Support
Gröhrmühlgasse 26, 2700 Wiener Neustadt
Tel: +43/1/205 10 35 / 340
www.postgresql.at, www.cybertec.at
Hans-Juergen Schoenig <hs@cybertec.at> writes:
Doing this for XIDs is pretty useless this days.
It is only targeted for command ids which are consumed heavily by
stored procedure languages.
It happens once on a while that a complex business logic procedure
runs out of command ids inside a transaction.
the idea is to give users a chance to avoid that.
touching XIDs does not make sense to me at all.
In view of the fact that 8.3 greatly reduced the CommandID consumption
of typical plpgsql code
http://archives.postgresql.org/pgsql-committers/2007-11/msg00585.php
I wonder whether the case for wider CIDs hasn't likewise taken a
major hit.
regards, tom lane