OID wraparound: summary and proposal

Started by Tom Laneover 24 years ago74 messageshackers
Jump to latest
#1Tom Lane
tgl@sss.pgh.pa.us

Given Hiroshi's objections, and the likelihood of compatibility problems
for existing applications, I am now thinking that it's not a good idea to
turn off OID generation by default. (At least not for 7.2 --- maybe in
some future release we could change the default.)

Based on the discussion so far, here is an attempt to flesh out the
details of what to do with OIDs for 7.2:

1. Add an optional clause "WITH OIDS" or "WITHOUT OIDS" to CREATE TABLE.
The default behavior will be WITH OIDS.

Note: there was some discussion of a GUC variable to control the default.
I'm leaning against this, mainly because having one would mean that
pg_dump *must* write WITH OIDS or WITHOUT OIDS in every CREATE TABLE;
else it couldn't be sure that the database schema would be correctly
reconstructed. That would create dump-script portability problems and
negate some of the point of having a GUC variable in the first place.
So I'm thinking a fixed default is better.

Note: an alternative syntax possibility is to make it look like the "with"
option clauses for functions and indexes: "WITH (oids)" or "WITH (noOids)".
This is uglier today, but would start to look more attractive if we invent
additional CREATE TABLE options in the future --- there'd be a place to
put 'em. Comments?

2. A child table will be forced to have OIDs if any of its parents do,
even if WITHOUT OIDS is specified in the child's CREATE command. This is
on the theory that the OID ought to act like an inherited column.

3. For a table without OIDs, no entry will be made in pg_attribute for
the OID column, so an attempt to reference the OID column will draw a
"no such column" error. (An alternative is to allow OID to read as nulls,
but it seemed that people preferred the error to be raised.)

4. When inserting into an OID-less table, the INSERT result string will
always show 0 for the OID.

5. A "relhasoids" boolean column will be added to pg_class to signal
whether a table has OIDs or not.

6. COPY out WITH OIDS will ignore the "WITH OIDS" specification if the
table has no OIDs. (Alternative possibility: raise an error --- is that
better?) COPY in WITH OIDS will silently drop the incoming OID values.

7. Physical tuple headers won't change. If no OIDs are assigned for a
particular table, the OID field in the header will be left zero.

8. OID generation will be disabled for those system tables that don't need
it --- pg_listener, pg_largeobject, and pg_attribute being some major
offenders that consume lots of OIDs.

9. To continue to support COMMENT ON COLUMN when columns have no OIDs,
pg_description will be modified so that its primary key is (object type,
object OID, column number) --- this also solves the problem that comments
break if there are duplicate OIDs in different system tables. The object
type is the OID of the system catalog in which the object OID appears.
The column number field will be zero for all object types except columns.
For a column comment, the object type and OID fields will refer to the
parent table, and column number will be nonzero.

10. pg_dump will be modified to do the appropriate things with OIDs. Are
there any other application programs that need to change?

We had also talked about adding an INSERT ... RETURNING feature to allow
applications to eliminate their dependence on looking at the OID returned
by an INSERT command. I think this is a good idea, but there are still
a number of unsolved issues about how it should interact with rules.
Accordingly, I'm not going to try to include it in this batch of work.

Comments?

regards, tom lane

#2Stephan Szabo
sszabo@megazone23.bigpanda.com
In reply to: Tom Lane (#1)
Re: OID wraparound: summary and proposal

On Wed, 1 Aug 2001, Tom Lane wrote:

Based on the discussion so far, here is an attempt to flesh out the
details of what to do with OIDs for 7.2:

1. Add an optional clause "WITH OIDS" or "WITHOUT OIDS" to CREATE TABLE.
The default behavior will be WITH OIDS.

Note: there was some discussion of a GUC variable to control the default.

Note: an alternative syntax possibility is to make it look like the "with"
option clauses for functions and indexes: "WITH (oids)" or "WITH (noOids)".
This is uglier today, but would start to look more attractive if we invent
additional CREATE TABLE options in the future --- there'd be a place to
put 'em. Comments?

I think a fixed default and placing it in parentheses are probably good
ideas.

3. For a table without OIDs, no entry will be made in pg_attribute for
the OID column, so an attempt to reference the OID column will draw a
"no such column" error. (An alternative is to allow OID to read as nulls,
but it seemed that people preferred the error to be raised.)

Okay, at least the foreign key stuff will need to change (since it does a
select oid), but I don't think it ever does anything with that except
check for existance, so I could probably make it select 1 as reasonable
replacement.

#3Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#1)
Re: OID wraparound: summary and proposal

Given Hiroshi's objections, and the likelihood of compatibility problems
for existing applications, I am now thinking that it's not a good idea to
turn off OID generation by default. (At least not for 7.2 --- maybe in
some future release we could change the default.)

This seems good. People with oid concerns usually have 1-2 huge tables
and the rest are small.

Based on the discussion so far, here is an attempt to flesh out the
details of what to do with OIDs for 7.2:

1. Add an optional clause "WITH OIDS" or "WITHOUT OIDS" to CREATE TABLE.
The default behavior will be WITH OIDS.

Makes sense.

Note: there was some discussion of a GUC variable to control the default.
I'm leaning against this, mainly because having one would mean that
pg_dump *must* write WITH OIDS or WITHOUT OIDS in every CREATE TABLE;
else it couldn't be sure that the database schema would be correctly
reconstructed. That would create dump-script portability problems and
negate some of the point of having a GUC variable in the first place.
So I'm thinking a fixed default is better.

Good point.

Note: an alternative syntax possibility is to make it look like the "with"
option clauses for functions and indexes: "WITH (oids)" or "WITH (noOids)".
This is uglier today, but would start to look more attractive if we invent
additional CREATE TABLE options in the future --- there'd be a place to
put 'em. Comments?

I don't like the parens. Looks ugly and I am not used to seeing them
used that way. I can imagine later using WITH NOOIDS, NOBIBBLE, BABBLE.
Maybe the syntax should be WITH OID, WITH NOOID?

2. A child table will be forced to have OIDs if any of its parents do,
even if WITHOUT OIDS is specified in the child's CREATE command. This is
on the theory that the OID ought to act like an inherited column.

Good point.

3. For a table without OIDs, no entry will be made in pg_attribute for
the OID column, so an attempt to reference the OID column will draw a
"no such column" error. (An alternative is to allow OID to read as nulls,
but it seemed that people preferred the error to be raised.)

Makes sense.

6. COPY out WITH OIDS will ignore the "WITH OIDS" specification if the
table has no OIDs. (Alternative possibility: raise an error --- is that
better?) COPY in WITH OIDS will silently drop the incoming OID values.

Obviously, the case here is that COPY WITH OIDS alone on a non-oid table
should throw an error, while pg_dump -o should work on a database with
mixed oid/non-oid. I think the right thing would be to have pg_dump
check pg_class.relhasoids and issue a proper COPY statement to match the
existing table.

7. Physical tuple headers won't change. If no OIDs are assigned for a
particular table, the OID field in the header will be left zero.

8. OID generation will be disabled for those system tables that don't need
it --- pg_listener, pg_largeobject, and pg_attribute being some major
offenders that consume lots of OIDs.

9. To continue to support COMMENT ON COLUMN when columns have no OIDs,
pg_description will be modified so that its primary key is (object type,
object OID, column number) --- this also solves the problem that comments
break if there are duplicate OIDs in different system tables. The object
type is the OID of the system catalog in which the object OID appears.
The column number field will be zero for all object types except columns.
For a column comment, the object type and OID fields will refer to the
parent table, and column number will be nonzero.

Sounds like a hack. I still prefer pg_attribute to have oids. Can we
have temp tables have no pg_attribute oids? A hack on a hack?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#3)
Re: OID wraparound: summary and proposal

Bruce Momjian <pgman@candle.pha.pa.us> writes:

6. COPY out WITH OIDS will ignore the "WITH OIDS" specification if the
table has no OIDs. (Alternative possibility: raise an error --- is that
better?) COPY in WITH OIDS will silently drop the incoming OID values.

Obviously, the case here is that COPY WITH OIDS alone on a non-oid table
should throw an error, while pg_dump -o should work on a database with
mixed oid/non-oid. I think the right thing would be to have pg_dump
check pg_class.relhasoids and issue a proper COPY statement to match the
existing table.

pg_dump clearly will need to do that, so it isn't really going to be the
issue. The question is what to do when a less-clueful app issues a COPY
WITH OIDS on an OID-less table. For input, I see no downside to just
ignoring the incoming OIDs. For output, I can see three reasonable
possibilities:

A. Pretend WITH OIDS wasn't mentioned. This might seem to be
"do the right thing", but a rather strong objection is that the
app will not get back the data it was expecting.

B. Return NULLs or 0s for the OIDs column.

C. Raise an error and refuse to do the copy at all.

C is probably the most conservative answer.

9. To continue to support COMMENT ON COLUMN when columns have no OIDs,
pg_description will be modified so that its primary key is (object type,
object OID, column number) --- this also solves the problem that comments
break if there are duplicate OIDs in different system tables. The object
type is the OID of the system catalog in which the object OID appears.
The column number field will be zero for all object types except columns.
For a column comment, the object type and OID fields will refer to the
parent table, and column number will be nonzero.

Sounds like a hack.

How so? pg_description is broken anyway given that we don't enforce OID
uniqueness across system catalogs. Also, in the future we could
consider overloading the <column number> column to have meanings for
other object types. I could imagine using it to attach documentation to
each of the input arguments of a function, for example.

regards, tom lane

#5Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#4)
Re: OID wraparound: summary and proposal

Bruce Momjian <pgman@candle.pha.pa.us> writes:

6. COPY out WITH OIDS will ignore the "WITH OIDS" specification if the
table has no OIDs. (Alternative possibility: raise an error --- is that
better?) COPY in WITH OIDS will silently drop the incoming OID values.

Obviously, the case here is that COPY WITH OIDS alone on a non-oid table
should throw an error, while pg_dump -o should work on a database with
mixed oid/non-oid. I think the right thing would be to have pg_dump
check pg_class.relhasoids and issue a proper COPY statement to match the
existing table.

pg_dump clearly will need to do that, so it isn't really going to be the
issue. The question is what to do when a less-clueful app issues a COPY
WITH OIDS on an OID-less table. For input, I see no downside to just
ignoring the incoming OIDs. For output, I can see three reasonable
possibilities:

A. Pretend WITH OIDS wasn't mentioned. This might seem to be
"do the right thing", but a rather strong objection is that the
app will not get back the data it was expecting.

B. Return NULLs or 0s for the OIDs column.

C. Raise an error and refuse to do the copy at all.

C is probably the most conservative answer.

If we fail on load, we should fail on dump. Why not fail on COPY WITH
OIDS on a non-oid table?

9. To continue to support COMMENT ON COLUMN when columns have no OIDs,
pg_description will be modified so that its primary key is (object type,
object OID, column number) --- this also solves the problem that comments
break if there are duplicate OIDs in different system tables. The object
type is the OID of the system catalog in which the object OID appears.
The column number field will be zero for all object types except columns.
For a column comment, the object type and OID fields will refer to the
parent table, and column number will be nonzero.

Sounds like a hack.

How so? pg_description is broken anyway given that we don't enforce OID
uniqueness across system catalogs. Also, in the future we could

We have a script to detect them and the oid counter it unique. In what
way do we not enforce it.

consider overloading the <column number> column to have meanings for
other object types. I could imagine using it to attach documentation to
each of the input arguments of a function, for example.

Interesting idea.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#5)
Re: OID wraparound: summary and proposal

Bruce Momjian <pgman@candle.pha.pa.us> writes:

For input, I see no downside to just
ignoring the incoming OIDs. For output, I can see three reasonable
possibilities:

A. Pretend WITH OIDS wasn't mentioned. This might seem to be
"do the right thing", but a rather strong objection is that the
app will not get back the data it was expecting.

B. Return NULLs or 0s for the OIDs column.

C. Raise an error and refuse to do the copy at all.

C is probably the most conservative answer.

If we fail on load, we should fail on dump. Why not fail on COPY WITH
OIDS on a non-oid table?

I'm confused --- I was proposing that we *not* fail on load. What's the
point of failing on load?

How so? pg_description is broken anyway given that we don't enforce OID
uniqueness across system catalogs. Also, in the future we could

We have a script to detect them and the oid counter it unique. In what
way do we not enforce it.

In a running system, once the OID counter wraps around there's no
guarantee that you won't have duplicate OIDs in different system
tables. The only enforcement mechanism we have is the unique indexes,
and those will only check per-table. However, that's fine --- it's
as much as we need. For everything except pg_description, that is.
Since pg_description currently makes an unchecked and uncheckable
assumption of global uniqueness of OIDs, it's broken.

regards, tom lane

#7Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: Tom Lane (#1)
Re: OID wraparound: summary and proposal

Tom Lane wrote:

Given Hiroshi's objections, and the likelihood of compatibility problems
for existing applications, I am now thinking that it's not a good idea to
turn off OID generation by default. (At least not for 7.2 --- maybe in
some future release we could change the default.)

Would OIDs be globally unique or per table ?

regards,
Hiroshi Inoue

#8Tom Lane
tgl@sss.pgh.pa.us
In reply to: Hiroshi Inoue (#7)
Re: OID wraparound: summary and proposal

Hiroshi Inoue <Inoue@tpf.co.jp> writes:

Tom Lane wrote:

Given Hiroshi's objections, and the likelihood of compatibility problems
for existing applications, I am now thinking that it's not a good idea to
turn off OID generation by default.

Would OIDs be globally unique or per table ?

Same as now: if you have a unique index on 'em, they're unique within a
table; otherwise, no guarantee at all (once the system wraps around).

We should document this state of affairs better, of course, but I'm not
proposing to change it. The point here is just to let people suppress
OIDs for tables that don't need them, and thereby postpone OID wraparound.

regards, tom lane

#9Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: Tom Lane (#1)
Re: OID wraparound: summary and proposal

Tom Lane wrote:

Hiroshi Inoue <Inoue@tpf.co.jp> writes:

Tom Lane wrote:

Given Hiroshi's objections, and the likelihood of compatibility problems
for existing applications, I am now thinking that it's not a good idea to
turn off OID generation by default.

Would OIDs be globally unique or per table ?

Same as now: if you have a unique index on 'em, they're unique within a
table; otherwise, no guarantee at all (once the system wraps around).

OIDs per table seems more important than others.

Strangely enough, I've seen no objection to optional OIDs
other than mine. Probably it was my mistake to have formulated
a plan on the flimsy assumption.

regards,
Hiroshi Inoue

#10mlw
markw@mohawksoft.com
In reply to: Tom Lane (#1)
Re: OID wraparound: summary and proposal

Maybe I'm being horribly stupid here, but....

If the thinking is that some tables can escape having an OID, thus meaning OIDs
can be controlled by table, how hard would it be to have an OID range on a per
table basis?

Where each table to have its own notion of an OID, then OID wrap/depletion
should be minimal.

Tom Lane wrote:

Given Hiroshi's objections, and the likelihood of compatibility problems
for existing applications, I am now thinking that it's not a good idea to
turn off OID generation by default. (At least not for 7.2 --- maybe in
some future release we could change the default.)

Based on the discussion so far, here is an attempt to flesh out the
details of what to do with OIDs for 7.2:

1. Add an optional clause "WITH OIDS" or "WITHOUT OIDS" to CREATE TABLE.
The default behavior will be WITH OIDS.

Note: there was some discussion of a GUC variable to control the default.
I'm leaning against this, mainly because having one would mean that
pg_dump *must* write WITH OIDS or WITHOUT OIDS in every CREATE TABLE;
else it couldn't be sure that the database schema would be correctly
reconstructed. That would create dump-script portability problems and
negate some of the point of having a GUC variable in the first place.
So I'm thinking a fixed default is better.

Note: an alternative syntax possibility is to make it look like the "with"
option clauses for functions and indexes: "WITH (oids)" or "WITH (noOids)".
This is uglier today, but would start to look more attractive if we invent
additional CREATE TABLE options in the future --- there'd be a place to
put 'em. Comments?

2. A child table will be forced to have OIDs if any of its parents do,
even if WITHOUT OIDS is specified in the child's CREATE command. This is
on the theory that the OID ought to act like an inherited column.

3. For a table without OIDs, no entry will be made in pg_attribute for
the OID column, so an attempt to reference the OID column will draw a
"no such column" error. (An alternative is to allow OID to read as nulls,
but it seemed that people preferred the error to be raised.)

4. When inserting into an OID-less table, the INSERT result string will
always show 0 for the OID.

5. A "relhasoids" boolean column will be added to pg_class to signal
whether a table has OIDs or not.

6. COPY out WITH OIDS will ignore the "WITH OIDS" specification if the
table has no OIDs. (Alternative possibility: raise an error --- is that
better?) COPY in WITH OIDS will silently drop the incoming OID values.

7. Physical tuple headers won't change. If no OIDs are assigned for a
particular table, the OID field in the header will be left zero.

8. OID generation will be disabled for those system tables that don't need
it --- pg_listener, pg_largeobject, and pg_attribute being some major
offenders that consume lots of OIDs.

9. To continue to support COMMENT ON COLUMN when columns have no OIDs,
pg_description will be modified so that its primary key is (object type,
object OID, column number) --- this also solves the problem that comments
break if there are duplicate OIDs in different system tables. The object
type is the OID of the system catalog in which the object OID appears.
The column number field will be zero for all object types except columns.
For a column comment, the object type and OID fields will refer to the
parent table, and column number will be nonzero.

10. pg_dump will be modified to do the appropriate things with OIDs. Are
there any other application programs that need to change?

We had also talked about adding an INSERT ... RETURNING feature to allow
applications to eliminate their dependence on looking at the OID returned
by an INSERT command. I think this is a good idea, but there are still
a number of unsolved issues about how it should interact with rules.
Accordingly, I'm not going to try to include it in this batch of work.

Comments?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

--
5-4-3-2-1 Thunderbirds are GO!
------------------------
http://www.mohawksoft.com

#11Tom Lane
tgl@sss.pgh.pa.us
In reply to: mlw (#10)
Re: OID wraparound: summary and proposal

mlw <markw@mohawksoft.com> writes:

how hard would it be to have an OID range on a per
table basis?

The existing OID generator is a system-wide counter, and couldn't
reasonably be expected to do something like that.

There was some talk of (in essence) eliminating the present OID
generator mechanism and giving each table its own sequence object for
generating per-table OIDs. It's an interesting thought, but I'm
concerned about the overhead involved. At the very least we'd need to
reimplement sequence objects in a lower-overhead fashion (eg, make 'em
rows in a pg_sequence table rather than free-standing almost-tables).

Might be worth doing someday, but I think it's orthogonal to what I'm
proposing at present. There'd still be a need to suppress OID
generation on tables that don't need OIDs and might have more than
4 billion inserts during their lifetime.

regards, tom lane

#12Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#6)
Re: OID wraparound: summary and proposal

Bruce Momjian <pgman@candle.pha.pa.us> writes:

For input, I see no downside to just
ignoring the incoming OIDs. For output, I can see three reasonable
possibilities:

A. Pretend WITH OIDS wasn't mentioned. This might seem to be
"do the right thing", but a rather strong objection is that the
app will not get back the data it was expecting.

B. Return NULLs or 0s for the OIDs column.

C. Raise an error and refuse to do the copy at all.

C is probably the most conservative answer.

If we fail on load, we should fail on dump. Why not fail on COPY WITH
OIDS on a non-oid table?

I'm confused --- I was proposing that we *not* fail on load. What's the
point of failing on load?

I meant to say we should fail on dump _and_ load. If we don't we are
throwing away the oid's they are loading because though the table has no
oid column. Seems like something that should fail.

How so? pg_description is broken anyway given that we don't enforce OID
uniqueness across system catalogs. Also, in the future we could

We have a script to detect them and the oid counter it unique. In what
way do we not enforce it.

In a running system, once the OID counter wraps around there's no
guarantee that you won't have duplicate OIDs in different system
tables. The only enforcement mechanism we have is the unique indexes,
and those will only check per-table. However, that's fine --- it's
as much as we need. For everything except pg_description, that is.
Since pg_description currently makes an unchecked and uncheckable
assumption of global uniqueness of OIDs, it's broken.

If you consider random table creation failures acceptible. In oid
wraparound, whether pg_description could point to two rows with the same
oid is the smallest part of our problem. I think the whole idea we can
run reliably with an oid wraparound is questionable.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#13Zeugswetter Andreas SB
ZeugswetterA@wien.spardat.at
In reply to: Bruce Momjian (#12)
AW: OID wraparound: summary and proposal

Strangely enough, I've seen no objection to optional OIDs
other than mine. Probably it was my mistake to have formulated
a plan on the flimsy assumption.

I for one am more concerned about adding additional per
tuple overhead (moving from 32 -> 64bit) than loosing OID's
on some large tables. Imho optional OID's is the best way to combine
both worlds. OID's only where you absolutely need them, and thus
a good chance that wraparound does not happen during the lifetime of
one application. (And all this by reducing overhead, and not adding
overhead :-)

Andreas

#14Bruce Momjian
bruce@momjian.us
In reply to: Zeugswetter Andreas SB (#13)
Re: AW: OID wraparound: summary and proposal

Strangely enough, I've seen no objection to optional OIDs
other than mine. Probably it was my mistake to have formulated
a plan on the flimsy assumption.

I for one am more concerned about adding additional per
tuple overhead (moving from 32 -> 64bit) than loosing OID's
on some large tables. Imho optional OID's is the best way to combine
both worlds. OID's only where you absolutely need them, and thus
a good chance that wraparound does not happen during the lifetime of
one application. (And all this by reducing overhead, and not adding
overhead :-)

Agreed, the big selling point for me and optional oid's was removing
their overhead from the tuple header. We need to trim that baby down!

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#15Hannu Krosing
hannu@tm.ee
In reply to: Tom Lane (#1)
Re: Re: OID wraparound: summary and proposal

Tom Lane wrote:

Given Hiroshi's objections, and the likelihood of compatibility problems
for existing applications, I am now thinking that it's not a good idea to
turn off OID generation by default. (At least not for 7.2 --- maybe in
some future release we could change the default.)

Based on the discussion so far, here is an attempt to flesh out the
details of what to do with OIDs for 7.2:

Also OIDS should be promoted to 8-byte integers at some future time.

1. Add an optional clause "WITH OIDS" or "WITHOUT OIDS" to CREATE TABLE.
The default behavior will be WITH OIDS.

Note: there was some discussion of a GUC variable to control the default.
I'm leaning against this, mainly because having one would mean that
pg_dump *must* write WITH OIDS or WITHOUT OIDS in every CREATE TABLE;
else it couldn't be sure that the database schema would be correctly
reconstructed. That would create dump-script portability problems and
negate some of the point of having a GUC variable in the first place.
So I'm thinking a fixed default is better.

Note: an alternative syntax possibility is to make it look like the "with"
option clauses for functions and indexes: "WITH (oids)" or "WITH (noOids)".
This is uglier today, but would start to look more attractive if we invent
additional CREATE TABLE options in the future --- there'd be a place to
put 'em. Comments?

2. A child table will be forced to have OIDs if any of its parents do,
even if WITHOUT OIDS is specified in the child's CREATE command. This is
on the theory that the OID ought to act like an inherited column.

3. For a table without OIDs, no entry will be made in pg_attribute for
the OID column, so an attempt to reference the OID column will draw a
"no such column" error. (An alternative is to allow OID to read as nulls,
but it seemed that people preferred the error to be raised.)

4. When inserting into an OID-less table, the INSERT result string will
always show 0 for the OID.

5. A "relhasoids" boolean column will be added to pg_class to signal
whether a table has OIDs or not.

6. COPY out WITH OIDS will ignore the "WITH OIDS" specification if the
table has no OIDs. (Alternative possibility: raise an error --- is that
better?) COPY in WITH OIDS will silently drop the incoming OID values.

7. Physical tuple headers won't change. If no OIDs are assigned for a
particular table, the OID field in the header will be left zero.

8. OID generation will be disabled for those system tables that don't need
it --- pg_listener, pg_largeobject, and pg_attribute being some major
offenders that consume lots of OIDs.

1-8 sounds good

9. To continue to support COMMENT ON COLUMN when columns have no OIDs,
pg_description will be modified so that its primary key is (object type,
object OID, column number) --- this also solves the problem that comments
break if there are duplicate OIDs in different system tables.

Hm.. To me this sounds like allowing duplicates in an unique index in
case
there happen to be duplicate keys there ;)

IMHO duplicate OID's in system tables should be treated as bug's - if
they
are there they are meant to break stuff.

The object
type is the OID of the system catalog in which the object OID appears.
The column number field will be zero for all object types except columns.
For a column comment, the object type and OID fields will refer to the
parent table, and column number will be nonzero.

What happens to columns added to inherited tables ?

Show quoted text

10. pg_dump will be modified to do the appropriate things with OIDs. Are
there any other application programs that need to change?

We had also talked about adding an INSERT ... RETURNING feature to allow
applications to eliminate their dependence on looking at the OID returned
by an INSERT command. I think this is a good idea, but there are still
a number of unsolved issues about how it should interact with rules.
Accordingly, I'm not going to try to include it in this batch of work.

#16Hannu Krosing
hannu@tm.ee
In reply to: Zeugswetter Andreas SB (#13)
Re: OID wraparound: summary and proposal

Nathan Myers wrote:

On Thu, Aug 02, 2001 at 09:28:18AM +0200, Zeugswetter Andreas SB wrote:

Strangely enough, I've seen no objection to optional OIDs
other than mine. Probably it was my mistake to have formulated
a plan on the flimsy assumption.

I for one am more concerned about adding additional per
tuple overhead (moving from 32 -> 64bit) than loosing OID's
on some large tables. Imho optional OID's is the best way to combine
both worlds.

At the same time that we announce support for optional OIDs,
we should announce that, in future releases, OIDs will only be
guaranteed unique (modulo wraparounds) within a single table.

What would the purpose of such an announcement be ???

OID is "Object IDentifier", meant to uniquely identify ANY object in an
Object-Relational Database ,which PostgreSQL sometimes claims itself to
be.

If they are unique only within a single table then they are just
system-supplied primary key fields without a default index - quite
useless IMHO

I hope someone takes up the task of putting back some of the
niftier features of original Postgres/postgres95 and adding more OO
features. Deprecating OIDs won't help there .

--------------------
Hannu

#17Hannu Krosing
hannu@tm.ee
In reply to: Tom Lane (#1)
Re: Re: OID wraparound: summary and proposal

Tom Lane wrote:

Hannu Krosing <hannu@tm.ee> writes:

9. To continue to support COMMENT ON COLUMN when columns have no OIDs,
pg_description will be modified so that its primary key is (object type,
object OID, column number) --- this also solves the problem that comments
break if there are duplicate OIDs in different system tables.

Hm.. To me this sounds like allowing duplicates in an unique index in
case there happen to be duplicate keys there ;)

Unless you want to implement a global unique index

Or insert/update trigger

that can enforce uniqueness of OIDs across all the system tables, I don't
think that approach is tenable.

As I wrote in another mail to this list, AFAIK OID is supposed to be
Object Identifier - something that can be used to identify any object
in a unique fashion.

When (and if ;) we will implement SQL3's UNDER, we should, IMHO, make a
primary
key inherited and *unique* over all tables created UNDER the main table,
meaning
that we will need a way to have uniqueness constraint spanning multiple
tables.

( At least logically multiple tables, as IMHO UNDER with its single
inheritance is
best implemented in a single table with a bit more flexible column
structure. )

At that time we could theoretically inherit all system tables that have
OID
column from table "pg_system(oid oid primary key);"

pg_description is broken as it stands. Bruce
doesn't like the "column number" part of my proposal --- I suppose he'd
rather see the pg_description key as just <object type, object OID> with
object type referring to pg_attribute if it's a comment on column.
That would work too as far as fixing the lack of uniqueness goes, but it
still leaves us with pg_attribute as a significant consumer of OIDs.

That would probably be a problem with 4-byte OIDs, there is an ample
supply of 8-byte ones

I do like dropping OID from pg_listener, as it is a mostly empty and
really
rapidly changing table, but I see little value in dropping oid from
pg_attribute.

BTW, don't indexes, triggers or saved plans use OIDs from pg_attribute ?

Since the major point of this exercise (in my mind) is cutting the rate
of consumption of OIDs to postpone wraparound, I want to suppress OIDs
in pg_attribute, and to do that I have to add the column number to
pg_description.

I still think that going to 8-byte OIDs would be the best use of your
time ;)

If you can make the size of oid's a compile time option, then even
better.

Postponing the wraparound by the means you describe may be a fools
errand anyway,
as there are other ways to quickly consume oids that are very likely as
common as
those involving pg_listener, pg_largeobject, and pg_attribute.

Also computers still get faster, and disks still get bigger at the rate
I doubt
you will be able to match by finding ways to postpone the wraparound.

So here I'd like to contradict Vadim's claim that the time of simple
solutions is
over for PostgreSQL - making OID bigger is at least conceptually simple,
it's just
"a small matter of programming" ;)

--------------
Hannu

#18Hannu Krosing
hannu@tm.ee
In reply to: Tom Lane (#1)
Re: Re: OID wraparound: summary and proposal

mlw wrote:

I posted this question earlier, but it looks like it never made it on.

If you can control the OIDs on a per table basis, and some tables need not even
have any, why not let each table have its own OID range? Essentially, each
record will be numbered relative to 0 on its table?

That would really cut down the OID wrap around problem, and allow records to
have a notion of serialization.

What would the meaning of such an "OID" be ?

Apart from duplicating the primary key that is ?

------------------
Hannu

#19Nathan Myers
ncm@zembu.com
In reply to: Zeugswetter Andreas SB (#13)
Re: OID wraparound: summary and proposal

On Thu, Aug 02, 2001 at 09:28:18AM +0200, Zeugswetter Andreas SB wrote:

Strangely enough, I've seen no objection to optional OIDs
other than mine. Probably it was my mistake to have formulated
a plan on the flimsy assumption.

I for one am more concerned about adding additional per
tuple overhead (moving from 32 -> 64bit) than loosing OID's
on some large tables. Imho optional OID's is the best way to combine
both worlds.

At the same time that we announce support for optional OIDs,
we should announce that, in future releases, OIDs will only be
guaranteed unique (modulo wraparounds) within a single table.

Nathan Myers
ncm@zembu.com

#20Tom Lane
tgl@sss.pgh.pa.us
In reply to: Nathan Myers (#19)
Re: OID wraparound: summary and proposal

ncm@zembu.com (Nathan Myers) writes:

At the same time that we announce support for optional OIDs,
we should announce that, in future releases, OIDs will only be
guaranteed unique (modulo wraparounds) within a single table.

Seems reasonable --- that will give people notice that we're thinking
about separate-OID-generator-per-table ideas.

Right now we don't really document any of these considerations,
but I plan to write something as part of the work I'm about to do.

regards, tom lane

#21Tom Lane
tgl@sss.pgh.pa.us
In reply to: Hannu Krosing (#15)
#22Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#1)
#23mlw
markw@mohawksoft.com
In reply to: Tom Lane (#1)
#24Tom Lane
tgl@sss.pgh.pa.us
In reply to: Hannu Krosing (#17)
#25Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: Zeugswetter Andreas SB (#13)
#26Tom Lane
tgl@sss.pgh.pa.us
In reply to: Hiroshi Inoue (#25)
#27Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: Zeugswetter Andreas SB (#13)
#28Tom Lane
tgl@sss.pgh.pa.us
In reply to: Hiroshi Inoue (#27)
#29mlw
markw@mohawksoft.com
In reply to: Tom Lane (#1)
#30mlw
markw@mohawksoft.com
In reply to: Tom Lane (#1)
#31mlw
markw@mohawksoft.com
In reply to: Tom Lane (#1)
#32Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#20)
#33Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#28)
#34Zeugswetter Andreas SB
ZeugswetterA@wien.spardat.at
In reply to: Bruce Momjian (#33)
#35Dave Cramer
pg@fastcrypt.com
In reply to: Bruce Momjian (#32)
#36Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: mlw (#30)
#37mlw
markw@mohawksoft.com
In reply to: Hiroshi Inoue (#36)
#38Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: Hiroshi Inoue (#36)
#39Zeugswetter Andreas SB SD
ZeugswetterA@spardat.at
In reply to: Hiroshi Inoue (#38)
#40mlw
markw@mohawksoft.com
In reply to: Zeugswetter Andreas SB SD (#39)
#41Zeugswetter Andreas SB SD
ZeugswetterA@spardat.at
In reply to: mlw (#40)
#42Alex Pilosov
alex@pilosoft.com
In reply to: mlw (#40)
#43mlw
markw@mohawksoft.com
In reply to: Zeugswetter Andreas SB SD (#41)
#44Alex Pilosov
alex@pilosoft.com
In reply to: mlw (#43)
#45Hannu Krosing
hannu@tm.ee
In reply to: Zeugswetter Andreas SB SD (#41)
#46Hannu Krosing
hannu@tm.ee
In reply to: Zeugswetter Andreas SB SD (#41)
#47Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: Alex Pilosov (#44)
#48Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alex Pilosov (#44)
#49mlw
markw@mohawksoft.com
In reply to: Hiroshi Inoue (#47)
#50Tom Lane
tgl@sss.pgh.pa.us
In reply to: mlw (#49)
#51Hannu Krosing
hannu@tm.ee
In reply to: Alex Pilosov (#44)
#52Hannu Krosing
hannu@tm.ee
In reply to: Hiroshi Inoue (#47)
#53Tatsuo Ishii
t-ishii@sra.co.jp
In reply to: Tom Lane (#1)
#54Tom Lane
tgl@sss.pgh.pa.us
In reply to: Hannu Krosing (#51)
#55Tom Lane
tgl@sss.pgh.pa.us
In reply to: Hannu Krosing (#52)
#56Fernando Nasser
fnasser@redhat.com
In reply to: Alex Pilosov (#44)
#57Bruce Momjian
bruce@momjian.us
In reply to: Tatsuo Ishii (#53)
#58Tom Lane
tgl@sss.pgh.pa.us
In reply to: Fernando Nasser (#56)
#59Hannu Krosing
hannu@tm.ee
In reply to: Alex Pilosov (#44)
#60Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#57)
#61Fernando Nasser
fnasser@redhat.com
In reply to: Alex Pilosov (#44)
#62Tatsuo Ishii
t-ishii@sra.co.jp
In reply to: Tom Lane (#60)
#63Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tatsuo Ishii (#62)
#64mlw
markw@mohawksoft.com
In reply to: Bruce Momjian (#57)
#65Jan Wieck
JanWieck@Yahoo.com
In reply to: Tom Lane (#63)
#66Hannu Krosing
hannu@tm.ee
In reply to: Alex Pilosov (#44)
#67Tom Lane
tgl@sss.pgh.pa.us
In reply to: Jan Wieck (#65)
#68mlw
markw@mohawksoft.com
In reply to: Zeugswetter Andreas SB SD (#39)
#69Matthew T. O'Connor
matthew@zeut.net
In reply to: Zeugswetter Andreas SB SD (#39)
#70Neil Padgett
npadgett@redhat.com
In reply to: Zeugswetter Andreas SB SD (#39)
#71Doug McNaught
doug@wireboard.com
In reply to: Zeugswetter Andreas SB SD (#39)
#72Tom Lane
tgl@sss.pgh.pa.us
In reply to: Neil Padgett (#70)
#73Tom Lane
tgl@sss.pgh.pa.us
In reply to: Matthew T. O'Connor (#69)
#74Dale Johnson
dale@zembu.com
In reply to: Doug McNaught (#71)