Status of new relation file naming

[ Charset KOI8-R unsupported, converting... ]

Where are we in subj?
Oid.version or UniqueId?
If noone is going to implement it soon then I'll have to
change code to OID file naming for WAL.

Well, we can move to one of these, but we need tools so people can find
the real table names that go with the files, and even then, it will
never be as good as the system we currently have.

My idea was to append a version number or oid on to the end of the file
name, and use that somehow.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

On Tue, 12 Sep 2000, Bruce Momjian wrote:

[ Charset KOI8-R unsupported, converting... ]

Where are we in subj?
Oid.version or UniqueId?
If noone is going to implement it soon then I'll have to
change code to OID file naming for WAL.

Well, we can move to one of these, but we need tools so people can find
the real table names that go with the files, and even then, it will
never be as good as the system we currently have.

I thought it was generally agreed that this wasn't a requirement, and that
is someone felt it was required in the future, like any open source
project, they could ante up the time to build it ...

On Tue, 12 Sep 2000, Bruce Momjian wrote:

[ Charset KOI8-R unsupported, converting... ]

Where are we in subj?
Oid.version or UniqueId?
If noone is going to implement it soon then I'll have to
change code to OID file naming for WAL.

Well, we can move to one of these, but we need tools so people can find
the real table names that go with the files, and even then, it will
never be as good as the system we currently have.

I thought it was generally agreed that this wasn't a requirement, and that
is someone felt it was required in the future, like any open source
project, they could ante up the time to build it ...

Well, if we release 7.1 without those tools, we can expect lots of
complaints.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

On Tue, 12 Sep 2000, Bruce Momjian wrote:

On Tue, 12 Sep 2000, Bruce Momjian wrote:

[ Charset KOI8-R unsupported, converting... ]

Where are we in subj?
Oid.version or UniqueId?
If noone is going to implement it soon then I'll have to
change code to OID file naming for WAL.

Well, we can move to one of these, but we need tools so people can find
the real table names that go with the files, and even then, it will
never be as good as the system we currently have.

I thought it was generally agreed that this wasn't a requirement, and that
is someone felt it was required in the future, like any open source
project, they could ante up the time to build it ...

Well, if we release 7.1 without those tools, we can expect lots of
complaints.

stock answer "we look forward to seeing patches to correct this
problem" :)

I thought it was generally agreed that this wasn't a requirement, and that
is someone felt it was required in the future, like any open source
project, they could ante up the time to build it ...

Well, if we release 7.1 without those tools, we can expect lots of
complaints.

stock answer "we look forward to seeing patches to correct this
problem" :)

The problem is that I have no idea how to suggest writing such a tool
that fits all needs.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

On Tue, 12 Sep 2000, Bruce Momjian wrote:

I thought it was generally agreed that this wasn't a requirement, and that
is someone felt it was required in the future, like any open source
project, they could ante up the time to build it ...

Well, if we release 7.1 without those tools, we can expect lots of
complaints.

stock answer "we look forward to seeing patches to correct this
problem" :)

The problem is that I have no idea how to suggest writing such a tool
that fits all needs.

IMHO, we have a choice ... we either move forward with a change that I
*believe* everyone agrees has to happen, which will prompt someone to come
up with a solution to (again, what I believe) is the only drawback ... or,
we don't implement while waiting for someone to come up with a solution
...

if we wait, again, IMHO, it will never happen, since there is no impetus
for someone to "fix the problem" ...

depending on what it takes to implement, if we implement it now, that
leaves ~1.5mos for someone to come up with a solution before release ...

stock answer "we look forward to seeing patches to correct this
problem" :)

The problem is that I have no idea how to suggest writing such a tool
that fits all needs.

IMHO, we have a choice ... we either move forward with a change that I
*believe* everyone agrees has to happen, which will prompt someone to come
up with a solution to (again, what I believe) is the only drawback ... or,
we don't implement while waiting for someone to come up with a solution
...

if we wait, again, IMHO, it will never happen, since there is no impetus
for someone to "fix the problem" ...

depending on what it takes to implement, if we implement it now, that
leaves ~1.5mos for someone to come up with a solution before release ...

Well, we are 18 days from beta. Do we think we can handle that at this
time? If so, let's do it. I guess my hope was that WAL could record
the file/version# name in its logs, and use those to find the files,
rather than making the file names themselves just numbers, but I know
some thought that there was a chick-and-egg problem in doing that.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

On Tue, 12 Sep 2000, Bruce Momjian wrote:

stock answer "we look forward to seeing patches to correct this
problem" :)

The problem is that I have no idea how to suggest writing such a tool
that fits all needs.

IMHO, we have a choice ... we either move forward with a change that I
*believe* everyone agrees has to happen, which will prompt someone to come
up with a solution to (again, what I believe) is the only drawback ... or,
we don't implement while waiting for someone to come up with a solution
...

if we wait, again, IMHO, it will never happen, since there is no impetus
for someone to "fix the problem" ...

depending on what it takes to implement, if we implement it now, that
leaves ~1.5mos for someone to come up with a solution before release ...

Well, we are 18 days from beta. Do we think we can handle that at this
time?

My feeling was that the implementation was pretty much decided upon, it
was just a matter of saying "go ahead, do it" ... so, "go ahead, do
it" ... we have 18 days before beta, and then another 30 or so until
release, which should have us releasing, what, around Jan 1, 2001? :)

depending on what it takes to implement, if we implement it now, that
leaves ~1.5mos for someone to come up with a solution before release ...

Well, we are 18 days from beta. Do we think we can handle that at this
time?

My feeling was that the implementation was pretty much decided upon, it
was just a matter of saying "go ahead, do it" ... so, "go ahead, do
it" ... we have 18 days before beta, and then another 30 or so until
release, which should have us releasing, what, around Jan 1, 2001? :)

Yes, Jan 1 would be my guess. My only question is whether we are making
things harder for administrators and whether putting some smarts in WAL
to keep readable file names would be easier.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

On Tue, 12 Sep 2000, Bruce Momjian wrote:

depending on what it takes to implement, if we implement it now, that
leaves ~1.5mos for someone to come up with a solution before release ...

Well, we are 18 days from beta. Do we think we can handle that at this
time?

My feeling was that the implementation was pretty much decided upon, it
was just a matter of saying "go ahead, do it" ... so, "go ahead, do
it" ... we have 18 days before beta, and then another 30 or so until
release, which should have us releasing, what, around Jan 1, 2001? :)

Yes, Jan 1 would be my guess. My only question is whether we are making
things harder for administrators and whether putting some smarts in WAL
to keep readable file names would be easier.

guess we'll find out ... considering what I've seen of Oracle and its
layouts, I doubt we're making anything harder then most are used to :)

now, guess the next question is ... who is the one implementing this and
how soon can we get it in place?

My idea was to append a version number or oid on to the end
of the file name, and use that somehow.

You'll lose all you would buy as soon as we'll begin to store many
relations in single file... and I would like to implement this in 7.1

Vadim

[ Charset ISO-8859-1 unsupported, converting... ]

My idea was to append a version number or oid on to the end
of the file name, and use that somehow.

You'll lose all you would buy as soon as we'll begin to store many
relations in single file... and I would like to implement this in 7.1

Yes, that is the key. You want to implement a new storage manager, so
if that is coming, we may as well take the hit now.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

"Mikheev, Vadim" wrote:

My idea was to append a version number or oid on to the end
of the file name, and use that somehow.

You'll lose all you would buy as soon as we'll begin to store many
relations in single file...

Perhaps we could then use the name of DATASPACE = filename ?

Or will the fact that some relations are stored in the same file
be completely invisible to the user ?

and I would like to implement this in 7.1

Will this new storage manager replace the current one or will one be
able to choose which storage manager to use (at compile time, at
startup, for each table)?

PostgreSQL started as an extensible ORDBMS, but IIRC at some stage
all other SMs were thrown out.

I don't think it would be a good idea to completely abandon the
notion of storage manager as a replacable component.

OTOH, the idea of storing single-inheritance hierarchies
(SQL3 CREATE UNDER) in one file would almost automatically get us
many benefits, like shared primary keys and automatic index inheriting.

--------------
Hannu

You have to tell us whether you plan to implement
a safe file rename in WAL ? If yes a simple filename
without version would be possible and better.

Andreas

At 09:44 13/09/00 +0300, Hannu Krosing wrote:

"Mikheev, Vadim" wrote:

My idea was to append a version number or oid on to the end
of the file name, and use that somehow.

You'll lose all you would buy as soon as we'll begin to store many
relations in single file...

Perhaps we could then use the name of DATASPACE = filename ?

I don't want to (re)^n ignite the the debate, but file naming has been
discussed many times before and my recollection of the result of the last
debate was that the names should either be random or OID based. It seems
that adding a different meaning to filenames at this stage would be a bad
idea, and doom us to repeat the file naming debate again in a year or two.

My vote is for a random number, and then someone can write the tools to
display the file info. I'll even volunteer to work on them...

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.B.N. 75 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 0500 83 82 82 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

You have to tell us whether you plan to implement
a safe file rename in WAL ? If yes a simple filename
without version would be possible and better.

What do you mean?

Vadim

Will this new storage manager replace the current one or will one be
able to choose which storage manager to use (at compile time, at
startup, for each table)?

This would be possible, but no way if new smgr will be overwriting one
(smgr nature affects access methods).

PostgreSQL started as an extensible ORDBMS, but IIRC at some stage
all other SMs were thrown out.

There was just one additional smgr for stable memory. If someone has
this feature in comp then he could try to resurrect it.

I don't think it would be a good idea to completely abandon the
notion of storage manager as a replacable component.

Smgr wrapper is still in place.

Vadim

My vote is for a random number, and then someone can write
the tools to display the file info. I'll even volunteer to
work on them...

Ok. If someone will decide to implement this please try to use
RelFileNode structure defined in storage/relfilenode.h.
It's just place holder for the moment - I needed in *some*
structure as "file pointer" in log records - so feel free
to change context.

Vadim

At 10:48 13/09/00 -0700, Mikheev, Vadim wrote:

My vote is for a random number, and then someone can write
the tools to display the file info. I'll even volunteer to
work on them...

Ok. If someone will decide to implement this please try to use
RelFileNode structure defined in storage/relfilenode.h.

Just to clarify, I was offering to write the file info tools, not the
changes to the file handling in PG. Although I would be willing to help in
the latter for obvious reasons.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.B.N. 75 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 0500 83 82 82 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

You have to tell us whether you plan to implement
a safe file rename in WAL ? If yes a simple filename
without version would be possible and better.

What do you mean?

The previous discussion we had where we concluded, that
an os rename of a file cannot be done without risc.
But that risc was imho acceptable to avoid the extra version in the filename

(a rename back to the old name could fail when the tx is supposed
to be rolled back).

Search the archive for "file rename sync".

My conlusion would be an oid only filename, or a mixture of
oid and tablename, where tablename can be wildcarded on a directory search,
since oid is already unique. No version in the name, we would do renames in
that case.

If I remember correctly a patch exists that does oid only filenames.

Andreas

My vote is for a random number, and then someone can write
the tools to display the file info. I'll even volunteer to
work on them...

What was the advantage of random number over oid [+version]
in the light that there is an extra field in pg_class for other smgrs ?
We surely want readable names for tablespace files, no ?

Andreas

At 10:00 14/09/00 +0200, Zeugswetter Andreas SB wrote:

My vote is for a random number, and then someone can write
the tools to display the file info. I'll even volunteer to
work on them...

What was the advantage of random number over oid [+version]
in the light that there is an extra field in pg_class for other smgrs ?

None other than it removes the temptation to write utilities that rely on
the internal representation of our data.

We surely want readable names for tablespace files, no ?

So long as we say from the outset that 'rename tablespace' must be done
outside of transactions, and while the database (or at least the tables
that it contains) are not available.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.B.N. 75 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 0500 83 82 82 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

Rename... Why would we need in rename with OID filenames?
Ok, let's start with OID (*without tablename prefix|suffix*) filenames
and we'll see later how it will work.

So, could someone implement OID filenames?
(Please use RelFileNode structure).

Vadim

"Mikheev, Vadim" wrote:

Rename... Why would we need in rename with OID filenames?

Andreas seems to refer to in place replacement of OID files e.g.
using your *relink*.

Regards.

Hiroshi Inoue

Rename... Why would we need in rename with OID filenames?

Andreas seems to refer to in place replacement of OID files e.g.
using your *relink*.

Sorry, I've messed things for myself.

Ok. In short, I vote for UNIQUE_ID (unrelated to pg_class.oid) file names.
I think that it's better to implement this (but neither OID nor OID.VERSION)
right now
because of this is like what we'll have in new smgr -
tablespace_id.relation_file_node.
Pg_class' OID is kind of logical things, totaly unrelated to the issue
how/where to
store relation file.

Please comment ASAP.

Vadim

Philip Warner mentioned about the advantage of random number.
It's exactly what I've wanted to say.

it removes the temptation to write utilities that rely on
the internal representation of our data.

It is preferable that file naming rule is encapsulated so that we
can change it without notice.

So, I assume that you vote YES on this subject? -:)
(As far as I remember, it was your idea).

Vadim

On Thu, 14 Sep 2000, Mikheev, Vadim wrote:

Rename... Why would we need in rename with OID filenames?

Andreas seems to refer to in place replacement of OID files e.g.
using your *relink*.

Sorry, I've messed things for myself.

Ok. In short, I vote for UNIQUE_ID (unrelated to pg_class.oid) file names.
I think that it's better to implement this (but neither OID nor OID.VERSION)
right now
because of this is like what we'll have in new smgr -
tablespace_id.relation_file_node.
Pg_class' OID is kind of logical things, totaly unrelated to the issue
how/where to
store relation file.

Please comment ASAP.

sounds perfect to me

-----Original Message-----
From: Mikheev, Vadim [mailto:vmikheev@SECTORBASE.COM]

Rename... Why would we need in rename with OID filenames?

Andreas seems to refer to in place replacement of OID files e.g.
using your *relink*.

Sorry, I've messed things for myself.

Ok. In short, I vote for UNIQUE_ID (unrelated to pg_class.oid) file names.
I think that it's better to implement this (but neither OID nor
OID.VERSION)
right now
because of this is like what we'll have in new smgr -
tablespace_id.relation_file_node.
Pg_class' OID is kind of logical things, totaly unrelated to the issue
how/where to
store relation file.

Please comment ASAP.

Philip Warner mentioned about the advantage of random number.
It's exactly what I've wanted to say.

it removes the temptation to write utilities that rely on
the internal representation of our data.

It is preferable that file naming rule is encapsulated so that we
can change it without notice.

Regards.

Hiroshi Inoue

So, I assume that you vote YES on this subject? -:)
(As far as I remember, it was your idea).

Yes.

UNIQUE_ID file names: Hiroshi, Marc, Vadim

We can use oids as unique ids, but these were another oids -:)

Vadim

-----Original Message-----
From: Mikheev, Vadim [mailto:vmikheev@SECTORBASE.COM]

Philip Warner mentioned about the advantage of random number.
It's exactly what I've wanted to say.

it removes the temptation to write utilities that rely on
the internal representation of our data.

It is preferable that file naming rule is encapsulated so that we
can change it without notice.

So, I assume that you vote YES on this subject? -:)
(As far as I remember, it was your idea).

Yes.

Hiroshi Inoue