Fixed directory locations in installs

Bruce Momjian wrote:

Also, Win32 installs are going to want to be more directory
independent than Unix.

Why?

Because Win32 can probe for the location of the binary, it seems it
should check to see if it can find libdir and sharedir own its own
and set those GUC values accordingly as part of initdb.

This is just going to open up the possibility of silently finding the
wrong files.

Peter Eisentraut wrote:

Bruce Momjian wrote:

Also, Win32 installs are going to want to be more directory
independent than Unix.

Why?

Because when I install Win32 stuff on my machine via an installer, it
says "Where do you want the files" and puts it in C:\ or C:\Program
Files or whatever I supply. We need to have that functionality.

Because Win32 can probe for the location of the binary, it seems it
should check to see if it can find libdir and sharedir own its own
and set those GUC values accordingly as part of initdb.

This is just going to open up the possibility of silently finding the
wrong files.

Yes, it does. We need to check if the directory actually exists. If
the files we need don't exist in there, we will throw an error anyway,
and they will have to use a flag to specify the location. Win32 is
going to install everything under one directory anyway, so it should
work fine in most cases.

The only problem case would be where you have an initdb binary that sits
in a directory tree where the ../share and ../lib have files that initdb
needs, but they don't match the initdb version or something, but that
seems like a wacked-out configuration by any measure.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Peter Eisentraut wrote:

Bruce Momjian wrote:

Also, Win32 installs are going to want to be more directory
independent than Unix.

Why?

Common practice, for one thing. Windows programs are typically
relocatable, and Windows admins regard programs that rely on hardcoded
paths very poorly indeed.

The usual location used by an installer is something like
%ProgramFiles%/progname or %ProgramFiles%/progname/version - the
definition of %ProgramFiles% is determined by the machine it is being
installed on, not by the installer, and certainly can't be set at
compile time. The machine might not even have a C: drive, for instance.

But this is not only a Windows issue, as Tom reminded us recently. If I
understood him correctly, there have been calls for relocatable
installations from other binary packagers.

Because Win32 can probe for the location of the binary, it seems it
should check to see if it can find libdir and sharedir own its own
and set those GUC values accordingly as part of initdb.

This is just going to open up the possibility of silently finding the
wrong files.

Maybe it could be improved by using more version markers?

cheers

andrew

Bruce Momjian wrote:

In other words:

#dynamic_library_path = '$libdir'

could maybe become:

#dynamic_library_path = '/usr/local/pgsql/lib'

Not sure I follow - dynamic_library_path is a colon-separated set of
paths. How will somone using $libdir have that resolved? ISTM we need to
have another GUC var which sets it, rather than have it compiled in and
not able to be overridden. It's not really an initdb-time thing either -
it's an install-time thing - so the installer could add the right path
to postgresql.conf.sample. Of course, if people move directories around
after installation they are asking to shoot themselves in the foot, but
that's true now anyway.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

Peter Eisentraut wrote:

This is just going to open up the possibility of silently finding the
wrong files.

Maybe it could be improved by using more version markers?

AFAICS the sharedir will already be sufficiently checked by means of
initdb's check on the postgres.bki version marker. In some sense, the
sharedir used by initdb is the *right* one for an installation by
definition --- I'm not even convinced that we should allow people to
fool with this after the fact. (However, it's probably not worth the
trouble to develop a non-GUC mechanism to transmit the setting from
initdb to backend.)

We could add a version-marker file to libdir, but it'd not really be a
sufficient check since people might copy their own shlibs in there from
a prior installation without recompiling; and as soon as someone adds
more directories to dynamic_library_path, all bets are off anyway.
We've seen the "wrong version of plpgsql.so" symptom often enough that
I've thought seriously of insisting on a backend-version marker embedded
right into shlibs loaded by the backend. This'd be easy enough if we
were willing to demand a source code addition in loadable modules, say
a macro
BACKEND_VERSION_MARKER
which'd compile to some sort of preinitialized global variable or constant
function returning a version string. I haven't been able to think of a
way to insert such a marker without source-level cooperation though.

regards, tom lane

I wrote:

AFAICS the sharedir will already be sufficiently checked by means of
initdb's check on the postgres.bki version marker. In some sense, the
sharedir used by initdb is the *right* one for an installation by
definition --- I'm not even convinced that we should allow people to
fool with this after the fact.

Actually, looking at the present contents of the sharedir, I'm not even
sure that the backend needs to access it at all. Most of the files in
there are used only by initdb. The only thing that seems needed after
initdb is unknown.pltcl, and I'd not have a problem with moving that to
libdir.

regards, tom lane

Tom Lane wrote:

I wrote:

AFAICS the sharedir will already be sufficiently checked by means of
initdb's check on the postgres.bki version marker. In some sense, the
sharedir used by initdb is the *right* one for an installation by
definition --- I'm not even convinced that we should allow people to
fool with this after the fact.

Actually, looking at the present contents of the sharedir, I'm not even
sure that the backend needs to access it at all. Most of the files in
there are used only by initdb. The only thing that seems needed after
initdb is unknown.pltcl, and I'd not have a problem with moving that to
libdir.

Are you aware timezone is now in /share, at least in Win32, and maybe on
Unix some day?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

Tom Lane wrote:

I wrote:

AFAICS the sharedir will already be sufficiently checked by means of
initdb's check on the postgres.bki version marker. In some sense, the
sharedir used by initdb is the *right* one for an installation by
definition --- I'm not even convinced that we should allow people to
fool with this after the fact.

Actually, looking at the present contents of the sharedir, I'm not even
sure that the backend needs to access it at all. Most of the files in
there are used only by initdb. The only thing that seems needed after
initdb is unknown.pltcl, and I'd not have a problem with moving that to
libdir.

Are you aware timezone is now in /share, at least in Win32, and maybe on
Unix some day?

We could move the timezone under /lib if that would make things easier.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Andrew Dunstan wrote:

Common practice, for one thing. Windows programs are typically
relocatable, and Windows admins regard programs that rely on
hardcoded paths very poorly indeed.

OK, but how can that work in general? How do other programs handle
this? I don't think we should design a solution that goes like "ok, if
we move that file from share to lib then we could get away with it for
now". That will only postpone the potential problems. There needs to
be a definite and fixed place where programs can go looking to find the
files they need. Maybe it should be stored in that registry thing?

Common practice, for one thing. Windows programs are typically
relocatable, and Windows admins regard programs that rely on
hardcoded paths very poorly indeed.

OK, but how can that work in general? How do other programs handle
this? I don't think we should design a solution that goes
like "ok, if
we move that file from share to lib then we could get away with it for
now". That will only postpone the potential problems. There needs to
be a definite and fixed place where programs can go looking to
find the
files they need. Maybe it should be stored in that registry thing?

To make it work more cross-platform, replace "that registry thing" with
"postgresql.conf". It's basically the same thing, except the registry
has a hierarchy model.

And yes, that's how most other programs handle it.

//Magnus

Magnus Hagander wrote:

To make it work more cross-platform, replace "that registry thing"
with "postgresql.conf". It's basically the same thing, except the
registry has a hierarchy model.

That only works as long as all the files we want to refer to are used by
the server. But how will psql find /etc/psqlrc, how will libpq find
pg_service.conf, how will pg_dumpall find pg_dump, etc.?

Peter Eisentraut wrote:

Andrew Dunstan wrote:

Common practice, for one thing. Windows programs are typically
relocatable, and Windows admins regard programs that rely on
hardcoded paths very poorly indeed.

OK, but how can that work in general? How do other programs handle
this? I don't think we should design a solution that goes like "ok, if
we move that file from share to lib then we could get away with it for
now". That will only postpone the potential problems. There needs to
be a definite and fixed place where programs can go looking to find the
files they need. Maybe it should be stored in that registry thing?

Not portable. Also, storing things in the registry is pretty much the
Windows equivalent of storing things in /etc, an idea which has lately
been treated with less than universal approval, and suffers from almost
exactly the same problems. In particular, we need to ensure that a) you
don't need to be root/Administrator to install, and b) you can reliably
have multiple installations coexisting.

cheers

andrew

Peter Eisentraut wrote:

Magnus Hagander wrote:

To make it work more cross-platform, replace "that registry thing"
with "postgresql.conf". It's basically the same thing, except the
registry has a hierarchy model.

That only works as long as all the files we want to refer to are used by
the server. But how will psql find /etc/psqlrc, how will libpq find
pg_service.conf, how will pg_dumpall find pg_dump, etc.?

Actually, postgresql.conf only works once they run initdb (and we have
PGDATA defined). We still have the open issue of how initdb finds
postgresql.conf.sample unless we embed the file in the initdb binary
(double-yuck).

For pg_dump on Unix, we hopefully have it in our path, but on Win32, we
will not.

For Win32, we could use the registry. For Unix, we can't use /etc
because we can't be sure we are root. Can we create a dot-file in the
user's home directory during install?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

Peter Eisentraut wrote:

Magnus Hagander wrote:

To make it work more cross-platform, replace "that registry thing"
with "postgresql.conf". It's basically the same thing, except the
registry has a hierarchy model.

That only works as long as all the files we want to refer to are used by
the server. But how will psql find /etc/psqlrc, how will libpq find
pg_service.conf, how will pg_dumpall find pg_dump, etc.?

Actually, postgresql.conf only works once they run initdb (and we have
PGDATA defined). We still have the open issue of how initdb finds
postgresql.conf.sample unless we embed the file in the initdb binary
(double-yuck).

For pg_dump on Unix, we hopefully have it in our path, but on Win32, we
will not.

For Win32, we could use the registry. For Unix, we can't use /etc
because we can't be sure we are root. Can we create a dot-file in the
user's home directory during install?

We can't be sure we are Administrator either.

Binaries can find other binaries the way they do now: look in our own
location, then in the path.

Other files could be found by looking in 1) as per commandline (e.g. -L
in initdb) 2) hardcoded location, 3) our-location/../share

No config files / registry entries should be necessary, AFAICS.

cheers

andrew

Andrew Dunstan wrote:

Binaries can find other binaries the way they do now: look in our own
location, then in the path.

No, we can't look into the path. We have no information that says that
anything useful pertaining to our installation is in the path.

Other files could be found by looking in 1) as per commandline (e.g.
-L in initdb) 2) hardcoded location, 3) our-location/../share

Nothing says that ../share contains anything useful. Maybe it's
../share/pgsql, or maybe ../share/postgresql, or maybe
../share/postgresql-7.4.2 or maybe it's elsewhere altogether because
you have just moved your installation tree to make room for a new one.
We can't take guesses like this based on "usual installations".

The only hard facts that we can use are hardcoded/compiled-in locations
and explicit information passed via command-line arguments or
environment variables. None of this seems to be useful for Windows
installations. As far as I recall, the Windows installation routines
only ask you for one installation directory but not all the individual
ones. If this is true, then we could hardcode relative paths, but
maybe I'm mistaken. Can someone give a couple of full examples of
typical Windows installation layouts?

Peter Eisentraut wrote:

Andrew Dunstan wrote:

Binaries can find other binaries the way they do now: look in our own
location, then in the path.

No, we can't look into the path. We have no information that says that
anything useful pertaining to our installation is in the path.

Well, assuming all the binaries are installed in one location our own
location should do the trick.

Other files could be found by looking in 1) as per commandline (e.g.
-L in initdb) 2) hardcoded location, 3) our-location/../share

Nothing says that ../share contains anything useful. Maybe it's
../share/pgsql, or maybe ../share/postgresql, or maybe
../share/postgresql-7.4.2 or maybe it's elsewhere altogether because
you have just moved your installation tree to make room for a new one.
We can't take guesses like this based on "usual installations".

The only hard facts that we can use are hardcoded/compiled-in locations
and explicit information passed via command-line arguments or
environment variables. None of this seems to be useful for Windows
installations. As far as I recall, the Windows installation routines
only ask you for one installation directory but not all the individual
ones. If this is true, then we could hardcode relative paths, but
maybe I'm mistaken. Can someone give a couple of full examples of
typical Windows installation layouts?

The only one I can think is typical is all in one location, e.g. as if
you had specified --prefix="c:/foo/postgresql"

But there might be more exotic animals out there.

cheers

andrew

Andrew Dunstan wrote:

For Win32, we could use the registry. For Unix, we can't use /etc
because we can't be sure we are root. Can we create a dot-file in the
user's home directory during install?

We can't be sure we are Administrator either.

Exactly. IMHO, using the registry is the worst possible solution, for the
reasons Andrew has already pointed out (ie. multiple installs, user
privileges, ...).

Binaries can find other binaries the way they do now: look in our own
location, then in the path.

Other files could be found by looking in 1) as per commandline (e.g. -L
in initdb) 2) hardcoded location, 3) our-location/../share

I concur (ie. on 2, the hard-coded configure location, which is
c:/msys/1.0/local/pgsql/<etc> on my box, will rarely point to anything
useful on a virgin user machine, and could point to the wrong thing on a
developers machine with multiple installs... hmmm).

I'm yet to see a convincing argument for why we can't adopt the
"binary-location/../share" approach as submitted late March. AFAICS, it was
rejected on the basis that it was not platform independent (no arguments
there) and that we could not rely on the ".." approach. Well, why not? It
would greatly simplify the Win32 installer as users need only nominate where
they want PostgreSQL to live (ie. all it has to do in this regard is dump
the entire pgsql directory structure under a single root, without any
config), and windows users who go and muck with the internal directory
structure of their installed apps don't generally expect their app to
continue working... :-)

The other thing to point out is that, given that it is reasonable to expect
that the vast majority of Windows users won't be rolling their own install,
a solution for this is *needed* for Win32, but is merely a desirable for the
other platforms.

If this idea is to be rejected on the grounds that we'd like a platform
independent solution, then I'd like to see discussion focused in this
regard.

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>

Claudio Natoli wrote:

I'm yet to see a convincing argument for why we can't adopt the
"binary-location/../share" approach as submitted late March. AFAICS,
it was rejected on the basis that it was not platform independent (no
arguments there) and that we could not rely on the ".." approach.

The only objection was that it hardcodes the layout already in the
source, which gives us no flexibility at all to try out different
installation layouts. If you want to compute the relative paths from
bindir to libdir etc. at build time based on actual configure options,
then I see no problem with that.

Peter Eisentraut wrote:

Claudio Natoli wrote:

I'm yet to see a convincing argument for why we can't adopt the
"binary-location/../share" approach as submitted late March. AFAICS,
it was rejected on the basis that it was not platform independent (no
arguments there) and that we could not rely on the ".." approach.

The only objection was that it hardcodes the layout already in the
source, which gives us no flexibility at all to try out different
installation layouts. If you want to compute the relative paths from
bindir to libdir etc. at build time based on actual configure
options, then I see no problem with that.

But we want to resolve the locations at run-time, not build or configure
time. For win32, I'm yet to see why this approach is egregious.

Do you have an alternative solution to propose?

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>