[Fwd: Re: [PATCHES] 64-bit CommandIds]

"Zoltan Boszormenyi" <zb@cybertec.at> writes:

Hi,

what's your opinion on this?
I saw response only from Alvaro on the -patches list.

I don't understand. The patch only affects configuration and SQL data type
code. It doesn't actually store the 64-bit commandid anywhere which would be
the actual hard part.

Do "phantom" command ids mean this all just works magically? Ie, the limit of
2^32 <cmin,cmax> pairs is still there but as long as you don't have to store
more than that many you get to have 2^64 raw ephemeral commandids?

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's Slony Replication support!

Gregory Stark wrote:

I don't understand. The patch only affects configuration and SQL data type
code. It doesn't actually store the 64-bit commandid anywhere which would be
the actual hard part.

Sure it does, this is the significant part of the patch:

*** pgsql.orig/src/include/c.h	2008-03-02 13:44:45.000000000 +0100
--- pgsql-cid64/src/include/c.h	2008-03-04 21:05:23.000000000 +0100
*************** typedef TransactionId MultiXactId;
*** 382,388 ****
--- 382,392 ----

typedef uint32 MultiXactOffset;

+ #ifdef USE_64BIT_COMMANDID
+ typedef uint64 CommandId;
+ #else
   typedef uint32 CommandId;
+ #endif

#define FirstCommandId ((CommandId) 0)

CommandId type is used in htup.h and elsewhere, which changes the
on-disk format.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Mar 10, 2008, at 12:06 PM, Heikki Linnakangas wrote:

Gregory Stark wrote:

I don't understand. The patch only affects configuration and SQL
data type
code. It doesn't actually store the 64-bit commandid anywhere
which would be
the actual hard part.

Sure it does, this is the significant part of the patch:

*** pgsql.orig/src/include/c.h	2008-03-02 13:44:45.000000000 +0100
--- pgsql-cid64/src/include/c.h	2008-03-04 21:05:23.000000000 +0100
*************** typedef TransactionId MultiXactId;
*** 382,388 ****
--- 382,392 ----

typedef uint32 MultiXactOffset;

+ #ifdef USE_64BIT_COMMANDID
+ typedef uint64 CommandId;
+ #else
typedef uint32 CommandId;
+ #endif

#define FirstCommandId ((CommandId) 0)

CommandId type is used in htup.h and elsewhere, which changes the
on-disk format.

If we're going to make this a ./configure option, ISTM we should do
the same with XID size as well. I know there are high-velocity
databases that could use that.
--
Decibel!, aka Jim C. Nasby, Database Architect decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828

"Decibel!" <decibel@decibel.org> writes:

If we're going to make this a ./configure option, ISTM we should do the same
with XID size as well. I know there are high-velocity databases that could use
that.

Keep in mind we just changed things so that read-only transactions don't
consume xids. That means you would have to be actually modifying 2-billion
records before wrap-around becomes an issue.

If you're modifying 2-billion records that quickly presumably you're going to
have other pressing reasons to run vacuum aside from xid freezing...

Also, consider that you're suggesting increasing the per-tuple overhead from
24 bytes to, if my arithmetic is right, 40 bytes.

So really you would need, say, a system with enough i/o bandwidth to handle
2-billion updates or inserts per day and with enough spare i/o bandwidth that
another 16-bytes on every one of those updates is ok, but without the ability
to run vacuum.

Also, we still have hope that the visibility map info will make running vacuum
even less of an imposition.

All that said I don't really see much reason not to make it an option. I just
don't think anyone really needs it. In 5-10 years though...

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's 24x7 Postgres support!

Gregory Stark <stark@enterprisedb.com> writes:

All that said I don't really see much reason not to make it an option. I just
don't think anyone really needs it. In 5-10 years though...

The manpower we'd have to invest in making it work and keeping it
working would be enough reason ...

regards, tom lane

"Decibel!" <decibel@decibel.org> writes:

If we're going to make this a ./configure option, ISTM we should
do the same
with XID size as well. I know there are high-velocity databases
that could use
that.

Keep in mind we just changed things so that read-only transactions
don't
consume xids. That means you would have to be actually modifying 2-
billion
records before wrap-around becomes an issue.

If you're modifying 2-billion records that quickly presumably
you're going to
have other pressing reasons to run vacuum aside from xid freezing...

Also, consider that you're suggesting increasing the per-tuple
overhead from
24 bytes to, if my arithmetic is right, 40 bytes.

So really you would need, say, a system with enough i/o bandwidth
to handle
2-billion updates or inserts per day and with enough spare i/o
bandwidth that
another 16-bytes on every one of those updates is ok, but without
the ability
to run vacuum.

Also, we still have hope that the visibility map info will make
running vacuum
even less of an imposition.

All that said I don't really see much reason not to make it an
option. I just
don't think anyone really needs it. In 5-10 years though...

Doing this for XIDs is pretty useless this days.
It is only targeted for command ids which are consumed heavily by
stored procedure languages.
It happens once on a while that a complex business logic procedure
runs out of command ids inside a transaction.
the idea is to give users a chance to avoid that.
touching XIDs does not make sense to me at all.

many thanks,

hans

--
Cybertec Schönig & Schönig GmbH
PostgreSQL Solutions and Support
Gröhrmühlgasse 26, 2700 Wiener Neustadt
Tel: +43/1/205 10 35 / 340
www.postgresql.at, www.cybertec.at

Hans-Juergen Schoenig <hs@cybertec.at> writes:

Doing this for XIDs is pretty useless this days.
It is only targeted for command ids which are consumed heavily by
stored procedure languages.
It happens once on a while that a complex business logic procedure
runs out of command ids inside a transaction.
the idea is to give users a chance to avoid that.
touching XIDs does not make sense to me at all.

In view of the fact that 8.3 greatly reduced the CommandID consumption
of typical plpgsql code
http://archives.postgresql.org/pgsql-committers/2007-11/msg00585.php
I wonder whether the case for wider CIDs hasn't likewise taken a
major hit.

regards, tom lane