pgsql-server/src backend/tcop/postgres.c backe ...

Thomas Lockhart wrote:

CVSROOT: /cvsroot
Module name: pgsql-server
Changes by: thomas@postgresql.org 02/08/04 02:26:38

Modified files:
src/backend/tcop: postgres.c
src/backend/bootstrap: bootstrap.c
src/backend/postmaster: postmaster.c
src/bin/initdb : initdb.sh
src/bin/pg_ctl : pg_ctl.sh
src/include/access: xlog.h

Log message:
Implement WAL log location control using "-X" or PGXLOG.

Woh, I didn't think we had agreement on this. This populates the 'X'
all over the system (postgres, postmaster, initdb, pg_ctl), while the
simple solution would be to add the flag only to initdb and use a
symlink from /data. I also think it is less error-prone because you
can't accidentally point to the wrong XLOG directory. In fact, you
almost have to use an environment variable unles you plan to specify -X
for all the commands. In my mind, PGDATA should take care of the whole
thing for pointing to your data.

If you want to do it this way, I request a vote.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Thomas Lockhart wrote:

Implement WAL log location control using "-X" or PGXLOG.

Woh, I didn't think we had agreement on this. This populates the 'X'
all over the system (postgres, postmaster, initdb, pg_ctl), while the
simple solution would be to add the flag only to initdb and use a
symlink from /data. I also think it is less error-prone because you
can't accidentally point to the wrong XLOG directory. In fact, you
almost have to use an environment variable unles you plan to specify -X
for all the commands. In my mind, PGDATA should take care of the whole
thing for pointing to your data.

If you want to do it this way, I request a vote.

I have to vote a strong NO on this. What the patch essentially does is
to decouple specification of the data directory ($PGDATA or -D) from the
xlog directory ($PGXLOG or -X). This might seem nice and clean and
symmetrical, but in fact it has no conceivable use except for shooting
yourself in the foot in a particularly nasty manner. The xlog *cannot*
be decoupled from the data directory --- there are pointers in
pg_control and in every single data page that depend on the state of
xlog. Trying to make them look independent is just a recipe for
breaking your database by starting the postmaster with the wrong
combination of PGDATA and PGXLOG. And I'm not at all sure it'll be
possible to recover after you do that: if the postmaster notices the
discrepancy, it is likely to try to fix it by rolling forward from the
last checkpoint it can find in the mismatching xlog. Oops :-(

I think the existing mechanism of using a symlink in the data directory
when you want to move xlog is far safer and more reliable. I do not see
what functionality is added by this patch that can possibly justify the
hazards it creates.

regards, tom lane

Can I propose a compromise? Thomas, can we have only initdb honor
PGXLOG, just like it honors PGDATA? That way, admins can set PGXLOG and
initdb will take care of the symlinking for them, and we don't have to
push PGXLOG/-X down into all the other server parts. How do people like
that?

As far as tablespaces, we have three ways of storing directory path
info: environment variables, in the database, or in symlinks. If we
choose the first or last options, XLOG will have to follow that plan.
If we choose database, well there is no database during initdb so XLOG
would have a different system of specifying the location. I think
having only initdb honor PGXLOG and using a symlink will give us maximum
flexibility when we decide on table spaces.

Actually, as I remember, there was a strong vote that tablespaces where
going to use symlinks for storage, at least in some way.

---------------------------------------------------------------------------

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Thomas Lockhart wrote:

Implement WAL log location control using "-X" or PGXLOG.

Woh, I didn't think we had agreement on this. This populates the 'X'
all over the system (postgres, postmaster, initdb, pg_ctl), while the
simple solution would be to add the flag only to initdb and use a
symlink from /data. I also think it is less error-prone because you
can't accidentally point to the wrong XLOG directory. In fact, you
almost have to use an environment variable unles you plan to specify -X
for all the commands. In my mind, PGDATA should take care of the whole
thing for pointing to your data.

If you want to do it this way, I request a vote.

I have to vote a strong NO on this. What the patch essentially does is
to decouple specification of the data directory ($PGDATA or -D) from the
xlog directory ($PGXLOG or -X). This might seem nice and clean and
symmetrical, but in fact it has no conceivable use except for shooting
yourself in the foot in a particularly nasty manner. The xlog *cannot*
be decoupled from the data directory --- there are pointers in
pg_control and in every single data page that depend on the state of
xlog. Trying to make them look independent is just a recipe for
breaking your database by starting the postmaster with the wrong
combination of PGDATA and PGXLOG. And I'm not at all sure it'll be
possible to recover after you do that: if the postmaster notices the
discrepancy, it is likely to try to fix it by rolling forward from the
last checkpoint it can find in the mismatching xlog. Oops :-(

I think the existing mechanism of using a symlink in the data directory
when you want to move xlog is far safer and more reliable. I do not see
what functionality is added by this patch that can possibly justify the
hazards it creates.

regards, tom lane

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Can I propose a compromise? Thomas, can we have only initdb honor
PGXLOG, just like it honors PGDATA? That way, admins can set PGXLOG and
initdb will take care of the symlinking for them, and we don't have to
push PGXLOG/-X down into all the other server parts.

That seems reasonable to me.

regards, tom lane

Thomas, have you commented on the objections to this patch? If so, I
didn't see it.

---------------------------------------------------------------------------

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Thomas Lockhart wrote:

Implement WAL log location control using "-X" or PGXLOG.

Woh, I didn't think we had agreement on this. This populates the 'X'
all over the system (postgres, postmaster, initdb, pg_ctl), while the
simple solution would be to add the flag only to initdb and use a
symlink from /data. I also think it is less error-prone because you
can't accidentally point to the wrong XLOG directory. In fact, you
almost have to use an environment variable unles you plan to specify -X
for all the commands. In my mind, PGDATA should take care of the whole
thing for pointing to your data.

If you want to do it this way, I request a vote.

I have to vote a strong NO on this. What the patch essentially does is
to decouple specification of the data directory ($PGDATA or -D) from the
xlog directory ($PGXLOG or -X). This might seem nice and clean and
symmetrical, but in fact it has no conceivable use except for shooting
yourself in the foot in a particularly nasty manner. The xlog *cannot*
be decoupled from the data directory --- there are pointers in
pg_control and in every single data page that depend on the state of
xlog. Trying to make them look independent is just a recipe for
breaking your database by starting the postmaster with the wrong
combination of PGDATA and PGXLOG. And I'm not at all sure it'll be
possible to recover after you do that: if the postmaster notices the
discrepancy, it is likely to try to fix it by rolling forward from the
last checkpoint it can find in the mismatching xlog. Oops :-(

I think the existing mechanism of using a symlink in the data directory
when you want to move xlog is far safer and more reliable. I do not see
what functionality is added by this patch that can possibly justify the
hazards it creates.

regards, tom lane

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Thomas, have you commented on the objections to this patch? If so, I
didn't see it.

Yes, there was quite a long thread on this.

- Thomas

Thomas, would you remind me of the concusions because I thought everyone
involved felt that it should be an initdb-only option, but I still see
it in CVS.

---------------------------------------------------------------------------

Thomas Lockhart wrote:

Thomas, have you commented on the objections to this patch? If so, I
didn't see it.

Yes, there was quite a long thread on this.

- Thomas

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thomas, would you remind me of the concusions because I thought everyone
involved felt that it should be an initdb-only option, but I still see
it in CVS.

?? "Concussions" as in brain bruises? ;)

I'm not sure I understand the question. I assume that we are talking
about the WAL log location feature I implemented recently. It is an
initdb-only option, and defaults to the current behavior *exactly*.

The new feature is to allow an argument to initdb to locate the WAL file
to another location. That location can be specified on the command line,
or through an environment variable. Neither form precludes use of the
other, and either form can be considered "best practice" depending on
your opinion of what that is.

The postmaster also recognizes the command line option and environment
variable. The only suggestion I got as an alternative involved soft
links, which is not portable, which is not robust, and which is not used
anywhere else in the system. If we moved toward relying on soft links
for distributing resources we will be moving in the wrong direction for
many reasons, some of which I've mentioned previously. GUC parameters
were also mentioned as a possibility, and the infrastructure does not
preclude that at any time.

I don't recall that there were very many folks "involved". There were
several opinions, though most were from folks who were not thinking of
implementing disk management features. Some opinions dealt with details,
and some seemed to deal with the wisdom of allowing anything other than
a "one partition" model of the database, which is nothing if not short
sighted. Current default behavior is as first implemented, and the new
feature allows locating the WAL logs in another area. For the current
state of the art, that seems competitive with features found in other
database products, and an essential step in teaching PostgreSQL to work
with very large databases.

I had thought to extend the capabilities to allow resource allocation
for individual tables and indices, which has *long* been identified as a
desired capability by folks who are managing large systems. It seemed
reasonable to have done in time for 7.3. I'm rethinking that, not
because it shouldn't happen, but because the process of discussing these
issues has become so argumentative, divisive, impolite, and unpleasant.
Which is a shame imho...

- Thomas

Thomas Lockhart wrote:

Thomas, would you remind me of the concusions because I thought everyone
involved felt that it should be an initdb-only option, but I still see
it in CVS.

?? "Concussions" as in brain bruises? ;)

Uh, conclusions. Sorry. New keyboard desk in new house. :-)

I'm not sure I understand the question. I assume that we are talking
about the WAL log location feature I implemented recently. It is an
initdb-only option, and defaults to the current behavior *exactly*.

Yep. What bothers me is the clutter to the other commands that allow
XLOG location specification when you would never really want to specify
it except as part of initdb. I just see those extra flags as
cruft/confusion.

Look at pg_ctl:

pg_ctl start [-w] [-D DATADIR] [-s] [-X PGXLOG] [-l FILENAME] [-o "OPTIONS"]

Which option doesn't make sense? -X. It is way beyond the
functionality of the command.

The new feature is to allow an argument to initdb to locate the WAL file
to another location. That location can be specified on the command line,
or through an environment variable. Neither form precludes use of the
other, and either form can be considered "best practice" depending on
your opinion of what that is.

The postmaster also recognizes the command line option and environment
variable. The only suggestion I got as an alternative involved soft
links, which is not portable, which is not robust, and which is not used
anywhere else in the system. If we moved toward relying on soft links
for distributing resources we will be moving in the wrong direction for
many reasons, some of which I've mentioned previously. GUC parameters
were also mentioned as a possibility, and the infrastructure does not
preclude that at any time.

I don't think anyone agreed with your concerns about symlinks. If you
want to be careful, do the "ln -s" in initdb and exit on failure, and
tell them not to use -X on that platform, though we use symlinks for
postmaster/postgres identification, so I know the only OS that doesn't
support symlinks is Netware, only because Netware folks just sent in a
patch to add a -post flag to work around lack of symlinks. (I have
asked for clarification from them.)

I actually requested a vote, and got several people who wanted my
compromise (PGXLOG or initdb -X flag only), and I didn't see anyone who
liked the addition of -X into non-initdb commands. Should I have a
specific vote? OK, three options:

1) -X, PGXLOG in initdb, postmaster, postgres, pg_ctl
2) -X, PGXLOG in initdb only
3) nothing

I remember a number of people liking 2, but we can vote again.

I don't recall that there were very many folks "involved". There were
several opinions, though most were from folks who were not thinking of
implementing disk management features. Some opinions dealt with details,
and some seemed to deal with the wisdom of allowing anything other than
a "one partition" model of the database, which is nothing if not short
sighted. Current default behavior is as first implemented, and the new
feature allows locating the WAL logs in another area. For the current
state of the art, that seems competitive with features found in other
database products, and an essential step in teaching PostgreSQL to work
with very large databases.

I had thought to extend the capabilities to allow resource allocation
for individual tables and indices, which has *long* been identified as a
desired capability by folks who are managing large systems. It seemed
reasonable to have done in time for 7.3. I'm rethinking that, not
because it shouldn't happen, but because the process of discussing these
issues has become so argumentative, divisive, impolite, and unpleasant.
Which is a shame imho...

I clearly want tablespaces, and it would be great for 7.3, and I don't
think it is a huge job. However, I think it will require symlinks to be
usable, and you probably do not, so it may be an issue.

As for the argumentativeness, we do have folks with some strong
opinions, and I guess on the PGXLOG issue, I am one of them. Maybe that
is bad? Are people expressing themselves badly? If so, I would like to
hear details either on list or privately so I can address them.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Fri, Aug 09, 2002 at 06:54:44PM -0700, Thomas Lockhart wrote:

I had thought to extend the capabilities to allow resource allocation
for individual tables and indices, which has *long* been identified as a
desired capability by folks who are managing large systems.

Without wishing to pour fuel on any fires, or advocate any
implementation, I can say for sure that this is very much a feature
we'd love to see.

A

-- 
----
Andrew Sullivan                               87 Mowat Avenue 
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M6K 3E3
                                         +1 416 646 3304 x110

OK, seeing as no one voted, and only Tom and I objected originally, we
will keep the code as Thomas has applied it, namely that PGXLOG/-X is
recognized by initdb, postmaster, postgres, and pg_ctl. There is no
symlink from the /data directory to the WAL location.

Thomas, you mentioned implementing table spaces. Are you planning to
use environment variables there too? You mentioned that Ingres's use of
environment variables wasn't well implemented. How will you improve on
it?

---------------------------------------------------------------------------

Bruce Momjian wrote:

Thomas Lockhart wrote:

Thomas, would you remind me of the concusions because I thought everyone
involved felt that it should be an initdb-only option, but I still see
it in CVS.

?? "Concussions" as in brain bruises? ;)

Uh, conclusions. Sorry. New keyboard desk in new house. :-)

I'm not sure I understand the question. I assume that we are talking
about the WAL log location feature I implemented recently. It is an
initdb-only option, and defaults to the current behavior *exactly*.

Yep. What bothers me is the clutter to the other commands that allow
XLOG location specification when you would never really want to specify
it except as part of initdb. I just see those extra flags as
cruft/confusion.

Look at pg_ctl:

pg_ctl start [-w] [-D DATADIR] [-s] [-X PGXLOG] [-l FILENAME] [-o "OPTIONS"]

Which option doesn't make sense? -X. It is way beyond the
functionality of the command.

The new feature is to allow an argument to initdb to locate the WAL file
to another location. That location can be specified on the command line,
or through an environment variable. Neither form precludes use of the
other, and either form can be considered "best practice" depending on
your opinion of what that is.

The postmaster also recognizes the command line option and environment
variable. The only suggestion I got as an alternative involved soft
links, which is not portable, which is not robust, and which is not used
anywhere else in the system. If we moved toward relying on soft links
for distributing resources we will be moving in the wrong direction for
many reasons, some of which I've mentioned previously. GUC parameters
were also mentioned as a possibility, and the infrastructure does not
preclude that at any time.

I don't think anyone agreed with your concerns about symlinks. If you
want to be careful, do the "ln -s" in initdb and exit on failure, and
tell them not to use -X on that platform, though we use symlinks for
postmaster/postgres identification, so I know the only OS that doesn't
support symlinks is Netware, only because Netware folks just sent in a
patch to add a -post flag to work around lack of symlinks. (I have
asked for clarification from them.)

I actually requested a vote, and got several people who wanted my
compromise (PGXLOG or initdb -X flag only), and I didn't see anyone who
liked the addition of -X into non-initdb commands. Should I have a
specific vote? OK, three options:

1) -X, PGXLOG in initdb, postmaster, postgres, pg_ctl
2) -X, PGXLOG in initdb only
3) nothing

I remember a number of people liking 2, but we can vote again.

I don't recall that there were very many folks "involved". There were
several opinions, though most were from folks who were not thinking of
implementing disk management features. Some opinions dealt with details,
and some seemed to deal with the wisdom of allowing anything other than
a "one partition" model of the database, which is nothing if not short
sighted. Current default behavior is as first implemented, and the new
feature allows locating the WAL logs in another area. For the current
state of the art, that seems competitive with features found in other
database products, and an essential step in teaching PostgreSQL to work
with very large databases.

I had thought to extend the capabilities to allow resource allocation
for individual tables and indices, which has *long* been identified as a
desired capability by folks who are managing large systems. It seemed
reasonable to have done in time for 7.3. I'm rethinking that, not
because it shouldn't happen, but because the process of discussing these
issues has become so argumentative, divisive, impolite, and unpleasant.
Which is a shame imho...

I clearly want tablespaces, and it would be great for 7.3, and I don't
think it is a huge job. However, I think it will require symlinks to be
usable, and you probably do not, so it may be an issue.

As for the argumentativeness, we do have folks with some strong
opinions, and I guess on the PGXLOG issue, I am one of them. Maybe that
is bad? Are people expressing themselves badly? If so, I would like to
hear details either on list or privately so I can address them.

-- 
Bruce Momjian                        |  http://candle.pha.pa.us
pgman@candle.pha.pa.us               |  (610) 359-1001
+  If your life is a hard drive,     |  13 Roberts Road
+  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

OK, seeing as no one voted, and only Tom and I objected originally, we
will keep the code as Thomas has applied it, namely that PGXLOG/-X is
recognized by initdb, postmaster, postgres, and pg_ctl.

We will? It looks to me like Thomas lost the vote 2-to-1.

Unless there are more votes, I'm going to *insist* that this code be
changed. It's dangerous and offers no offsetting benefit. XLOG
location should be settable at initdb, noplace later.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

OK, seeing as no one voted, and only Tom and I objected originally, we
will keep the code as Thomas has applied it, namely that PGXLOG/-X is
recognized by initdb, postmaster, postgres, and pg_ctl.

We will? It looks to me like Thomas lost the vote 2-to-1.

Unless there are more votes, I'm going to *insist* that this code be
changed. It's dangerous and offers no offsetting benefit. XLOG
location should be settable at initdb, noplace later.

Well, you didn't vote again in my follow up email, so I thought you
didn't care anymore, and Thomas didn't reply to my email either. I am
clearly concerned, as you are, but Thomas isn't, and no one else seems
to care.

Can two guys override another guy if he is doing the work? I usually
like to have a larger margin than that. I don't know what to do.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Tue, 13 Aug 2002, Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

OK, seeing as no one voted, and only Tom and I objected originally, we
will keep the code as Thomas has applied it, namely that PGXLOG/-X is
recognized by initdb, postmaster, postgres, and pg_ctl.

We will? It looks to me like Thomas lost the vote 2-to-1.

Unless there are more votes, I'm going to *insist* that this code be
changed. It's dangerous and offers no offsetting benefit. XLOG
location should be settable at initdb, noplace later.

Okay, I'm going to pop up here and side with Thomas ... I think ... I may
have missed some issues here, but, quite frankly, I hate symlinks, as I've
seen it create more evil then good .. hardlinks is a different story ...

And for that reason, I will side with Thomas ...

initdb should allow you to specify a seperate location, which I don't
think anyone disagrees with ... but, what happens if, for some reason, I
have to move it to another location? I have to then dump/reload after
doing a new initdb?

One thought at the back of my mind is why not have something like a
'PG_VERSION' for XLOG? Maybe something so simple as a text file in both
the data and xlog directory that just contains a timestamp from the
initdb? then, when you startup postmaster with a -X option, it compares
the two files and makes sure that they belong to each other?

Bruce, if I'm missing something, could you post a summary/scorecard for
pros-cons on this issue?

On Mon, 2002-08-12 at 23:09, Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

OK, seeing as no one voted, and only Tom and I objected originally, we
will keep the code as Thomas has applied it, namely that PGXLOG/-X is
recognized by initdb, postmaster, postgres, and pg_ctl.

We will? It looks to me like Thomas lost the vote 2-to-1.

Unless there are more votes, I'm going to *insist* that this code be
changed. It's dangerous and offers no offsetting benefit. XLOG
location should be settable at initdb, noplace later.

I think Tom is on to something here. I meant to ask but never got
around to it. Why would anyone need to move the XLOG after you've
inited the db?

Greg

On Tue, 13 Aug 2002, Bruce Momjian wrote:

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

OK, seeing as no one voted, and only Tom and I objected originally, we
will keep the code as Thomas has applied it, namely that PGXLOG/-X is
recognized by initdb, postmaster, postgres, and pg_ctl.

We will? It looks to me like Thomas lost the vote 2-to-1.

Unless there are more votes, I'm going to *insist* that this code be
changed. It's dangerous and offers no offsetting benefit. XLOG
location should be settable at initdb, noplace later.

Well, you didn't vote again in my follow up email, so I thought you
didn't care anymore, and Thomas didn't reply to my email either. I am
clearly concerned, as you are, but Thomas isn't, and no one else seems
to care.

k, why are you concerned? see my other message, but if the major concern
is the xlog directory for a postmaster, then put in a consistency check
for when starting ...

then again, how many out there are running multiple instances of
postmaster on their machine that would move xlog to a different location
without realizing they did? I know in my case, we're working at reducing
the # of instances on our servers to 2 (internal vs external), but, our
servers are all on a RAID5 array, so there is nowhere to really "move"
xlog to out then its default location ... but I can see the usefulness of
doing so ...

Myself, if I'm going to move something around, it will be after the server
has been running for a while and I've added in more drive space in order
to correct a problem i didn't anticipate (namely, disk I/O) ... at that
point, I really don't wan tto have to dump/re-initdb/load just to move the
xlog directory to that new drive ...

On 13 Aug 2002, Greg Copeland wrote:

On Mon, 2002-08-12 at 23:09, Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

OK, seeing as no one voted, and only Tom and I objected originally, we
will keep the code as Thomas has applied it, namely that PGXLOG/-X is
recognized by initdb, postmaster, postgres, and pg_ctl.

We will? It looks to me like Thomas lost the vote 2-to-1.

Unless there are more votes, I'm going to *insist* that this code be
changed. It's dangerous and offers no offsetting benefit. XLOG
location should be settable at initdb, noplace later.

I think Tom is on to something here. I meant to ask but never got
around to it. Why would anyone need to move the XLOG after you've
inited the db?

I just determined that disk I/O is terrible, so want to move the XLOG over
to a different file system that is currently totally idle ...

Marc G. Fournier wrote:

On Tue, 13 Aug 2002, Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

OK, seeing as no one voted, and only Tom and I objected originally, we
will keep the code as Thomas has applied it, namely that PGXLOG/-X is
recognized by initdb, postmaster, postgres, and pg_ctl.

We will? It looks to me like Thomas lost the vote 2-to-1.

Unless there are more votes, I'm going to *insist* that this code be
changed. It's dangerous and offers no offsetting benefit. XLOG
location should be settable at initdb, noplace later.

Okay, I'm going to pop up here and side with Thomas ... I think ... I may
have missed some issues here, but, quite frankly, I hate symlinks, as I've
seen it create more evil then good .. hardlinks is a different story ...

OK, that agrees with Thomas's aversion to symlinks.

And for that reason, I will side with Thomas ...

initdb should allow you to specify a seperate location, which I don't
think anyone disagrees with ... but, what happens if, for some reason, I
have to move it to another location? I have to then dump/reload after
doing a new initdb?

If you move pg_xlog, you have to create a symlink in /data that points
to the new location. Initdb would do that automatically, but if you
move it after initdb, you would have to create the symlink yourself.
With Thomas's current code, you would add/change PGXLOG instead to point
to the new location, rather than modify the symlink.

One thought at the back of my mind is why not have something like a
'PG_VERSION' for XLOG? Maybe something so simple as a text file in both
the data and xlog directory that just contains a timestamp from the
initdb? then, when you startup postmaster with a -X option, it compares
the two files and makes sure that they belong to each other?

Uh, seems it could get messy, but, yea, that would work. It means
adding a file to pg_xlog and /data and somehow matching them. My
feeling was that the symlink was unambiguous and allowed for fewer
mistakes. I think that was Tom's opinion too.

Bruce, if I'm missing something, could you post a summary/scorecard for
pros-cons on this issue?

OK, I tried. :-)

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Marc G. Fournier wrote:

Well, you didn't vote again in my follow up email, so I thought you
didn't care anymore, and Thomas didn't reply to my email either. I am
clearly concerned, as you are, but Thomas isn't, and no one else seems
to care.

k, why are you concerned? see my other message, but if the major concern
is the xlog directory for a postmaster, then put in a consistency check
for when starting ...

I really have two concerns; one, the ability to point to the wrong
place or to the default place accidentally if PGXLOG isn't set, and two,
the addition of PGXLOG/-X into postmaster, postgres, pg_ctl where it
really isn't adding any functionality and just adds bloat to the
commands arg list.

then again, how many out there are running multiple instances of
postmaster on their machine that would move xlog to a different location
without realizing they did? I know in my case, we're working at reducing
the # of instances on our servers to 2 (internal vs external), but, our
servers are all on a RAID5 array, so there is nowhere to really "move"
xlog to out then its default location ... but I can see the usefulness of
doing so ...

Myself, if I'm going to move something around, it will be after the server
has been running for a while and I've added in more drive space in order
to correct a problem i didn't anticipate (namely, disk I/O) ... at that
point, I really don't wan tto have to dump/re-initdb/load just to move the
xlog directory to that new drive ...

No dump/reload required, but the creation of a _dreaded_ symlink is
required.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073