pgxs default installation + various fixes

Am Donnerstag, 2. September 2004 10:16 schrieb Fabien COELHO:

(1) "make intall" installs everything.

(2) "make light-install" does not install pgxs and server dev stuff.
this is the previous version of "make install".

If we do that, we should remove install-all-headers. It's very confusing
otherwise.

This target maybe of interest of packagers.

This I don't understand.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Dear Peter,

(2) "make light-install" does not install pgxs and server dev stuff.
this is the previous version of "make install".

If we do that, we should remove install-all-headers. It's very confusing
otherwise.

This target maybe of interest of packagers.

This I don't understand.

I meant: "This target may be of interest for packagers", sorry for my poor
English writing and typing skills.

This is to take into account Tom's view that distributions are often split
with a separate "dev" package for developpements. For instance under
Debian there are: postgresql, postgresql-client, postgresql-dev,
postgresql-doc... So this target would help checking what files belong to
what part of such split installation for the package maintainer. I guess
that could possibly help administrators that would like to install a
small-footprint database.

Now, if everyone agree that having a distinct installation for server
headers and other stuff is useless, then indeed we can just keep the
install target and drop "install-all-headers" altogether, I agree with
you. That would indeed simplify the patch and the various makefiles,
and be less confusing.

If there is a consensus, I can remove both "light-install" and "install
-all-headers" targets in another submission.

Have a nice day,

--
Fabien Coelho - coelho@cri.ensmp.fr

On Thu, Sep 02, 2004 at 02:10:13PM +0200, Peter Eisentraut wrote:

Am Donnerstag, 2. September 2004 10:16 schrieb Fabien COELHO:

(1) "make intall" installs everything.

(2) "make light-install" does not install pgxs and server dev stuff.
this is the previous version of "make install".

If we do that, we should remove install-all-headers. It's very confusing
otherwise.

Why not get rid of install-all-headers and light-install? Just have one
target that installs all that's needed, and call it "install". Are we
trying to save disk space or what?

People who are short on disk space because they are trying to run on a
small device has a lot of manual work to do anyway.

This target maybe of interest of packagers.

This I don't understand.

I think the argument is that packagers may want to have different make
targets for the main package and the "development" package. AFAIK,
actually they don't.

What was the argument for having an "install-all-headers" target in the
first place? It creates a lot of unnecessary pain, just to save some
disk space.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"I think my standards have lowered enough that now I think 'good design'
is when the page doesn't irritate the living f*ck out of me." (JWZ)

On Thu, Sep 02, 2004 at 02:31:14PM +0200, Fabien COELHO wrote:

This is to take into account Tom's view that distributions are often split
with a separate "dev" package for developpements. For instance under
Debian there are: postgresql, postgresql-client, postgresql-dev,
postgresql-doc... So this target would help checking what files belong to
what part of such split installation for the package maintainer.

What they do AFAIK is to run the biggest install target and then
classify in different packages using file and glob lists hardcoded in
the package specification. Having more make targets does not help them.

At least that's how RPM packages are done.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"I call it GNU/Linux. Except the GNU/ is silent." (Ben Reiter)

Am Donnerstag, 2. September 2004 14:31 schrieb Fabien COELHO:

I meant: "This target may be of interest for packagers", sorry for my poor
English writing and typing skills.

I understood that. :)

This is to take into account Tom's view that distributions are often split
with a separate "dev" package for developpements. For instance under
Debian there are: postgresql, postgresql-client, postgresql-dev,
postgresql-doc... So this target would help checking what files belong to
what part of such split installation for the package maintainer.

No, the way this works is that the package building script installs everything
in one tree and then has file lists (with wildcards) that determine which
files belong in which package.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Dear Peter,

This is to take into account Tom's view that distributions are often split
with a separate "dev" package for developpements. For instance under
Debian there are: postgresql, postgresql-client, postgresql-dev,
postgresql-doc... So this target would help checking what files belong to
what part of such split installation for the package maintainer.

No, the way this works is that the package building script installs
everything in one tree and then has file lists (with wildcards) that
determine which files belong in which package.

Ok.

Thus it suggests that the only use for the two targets is whether someone
is interested in having a smaller footprint version...

The other reason why I sticked to it in the submission is to be upward
compatible with the previous state with 2 differents targets. I feel it is
a "political" decision to shift to a single target install, and I'm not
the one to take such a decision... I'm only really interested in having
server headers and pgxs stuff installed by default, so that extensions can
be reliably added on a default installation.

So, the question remains the same: is there a consensus to drop
install-all-headers/light-install targets for a simple and full "install"?

If so, I'll make a new submission.

--
Fabien Coelho - coelho@cri.ensmp.fr

Alvaro Herrera <alvherre@dcc.uchile.cl> writes:

What was the argument for having an "install-all-headers" target in the
first place?

By my count it adds about 2 megabytes to the installed footprint, which
perhaps is not so much now as it was at the time.

A larger problem from my point of view is that the installation process
seems quite inefficient. "make install-all-headers" takes a full minute
on my devel machine, which is above my threshold of pain for something
that I often do more than once a day.

regards, tom lane

Dear Tom, dear Peter,

By my count it adds about 2 megabytes to the installed footprint, which
perhaps is not so much now as it was at the time.

Ok. I agree. I'm investigating on how to do that, and the choice will be
to committers.

A larger problem from my point of view is that the installation process
seems quite inefficient. "make install-all-headers" takes a full minute
on my devel machine, which is above my threshold of pain for something
that I often do more than once a day.

As I'm into these files, I can say that one of the reason for that is that
the shell scripts in the makefile looks inefficient, with nested for-loops
and one-at-a-time config/install-sh forked-script copies for 350 header
files, on the 971 files of a standard installation.

I guess it a deliberate portability choice that only standard shell and
file wildcards are used. I'm not sure there is a simple and *portable*
performance fix on the installation.

Some files are copied more that once: for instance, pg_config.h is copied
both in include/ and include/server, and there are others examples file
that. In fact, all files and directories under include/ but include/server
seems also copied into include/server. It seems to be a 200KB and 28 files
replication. Not really big deal, but not really clean either.

If there is only one installation target which includes all header files,
ISTM that the include/server subdirectory does not make much sense
anymore? On the other hand, it might help packagers to have a clear list
of the use of all headers files, in order that they can figure out where
they belong to.

Opinions?

--
Fabien Coelho - coelho@cri.ensmp.fr

On Fri, Sep 03, 2004 at 10:42:30AM +0200, Fabien COELHO wrote:

As I'm into these files, I can say that one of the reason for that is that
the shell scripts in the makefile looks inefficient, with nested for-loops
and one-at-a-time config/install-sh forked-script copies for 350 header
files, on the 971 files of a standard installation.

Also the install-sh script apparently is way more complex than it needs
to be. There's probably a lot of that complexity (and subsequent
slowness) that install-all-headers doesn't need. A lot of time goes
into processing the script itself rather than doing useful work.

Is there an objection to trying to convert it to a simpler, faster
alternative? Maybe even one that receives multiple files as arguments,
which would reduce the number of times it is called by an order of
magnitude.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
<Schwern> It does it in a really, really complicated way
<crab> why does it need to be complicated?
<Schwern> Because it's MakeMaker.

Dear Alvaro,

As I'm into these files, I can say that one of the reason for that is that
the shell scripts in the makefile looks inefficient, with nested for-loops
and one-at-a-time config/install-sh forked-script copies for 350 header
files, on the 971 files of a standard installation.

Also the install-sh script apparently is way more complex than it needs
to be.

Maybe. I guess the options are there because they might be useful
sometimes?

There's probably a lot of that complexity (and subsequent slowness) that
install-all-headers doesn't need.

Maybe. I don't have a clear view about portability issues that I guess
justify this script.

A lot of time goes into processing the script itself rather than doing
useful work.

Yes.

Is there an objection to trying to convert it to a simpler, faster
alternative? Maybe even one that receives multiple files as arguments,
which would reduce the number of times it is called by an order of
magnitude.

Yes, handling several files at a time could indeed improve the stuff.
But this means changing the syntax somehow, and fixing makefiles...

Also, most unix box have an "install" program which might be more
efficient and which handles several files. I do not know whether it
has all the required facilities.

For instance, apache looks for a "bsd" install program at configuration
time... and a slow but compatible shell substitute is used instead if none
is available. Maybe this can be reused quite simply by postgresql, with
their kind permission. As apache is quite portable, it might be good
enough for pg.

This seems a reasonnable todo objective, but I'm not sure it should be
done for 8.0 as it changes the installation procedure significantly? Well,
maybe it could be done quite quiclky. It does not look too difficult to
implement with apache example at hand.

--
Fabien Coelho - coelho@cri.ensmp.fr

Fabien COELHO <coelho@cri.ensmp.fr> writes:

For instance, apache looks for a "bsd" install program at configuration
time... and a slow but compatible shell substitute is used instead if none
is available.

We used to do that, and gave up on it because there was too much
variability in the available install programs.

regards, tom lane

Where are we on this?

---------------------------------------------------------------------------

Fabien COELHO wrote:

Dear patchers,

This patch addresses open item "make pgxs install by default" in Bruce's
list.

(1) "make intall" installs everything.

(2) "make light-install" does not install pgxs and server dev stuff.
this is the previous version of "make install". This target maybe
of interest of packagers. If you want any other name, just tell
me what you want!

(3) the installation.sgml documentation reflects the new status...
but only with my poor English.

(4) I noticed that some pgxs related files were incidently installed
under "light-install", so I make sure that they are not.

(5) a minor stylistic fix in the pgxs makefile, so that it does not
have a double slash (/some/directory//sub/directory) in some paths.

What might be consider for future improvements,
but ISTM not mandatory for 8.0?

- maybe rename "install-all-headers" target, as it also install
scripts, makefiles and libraries useful for server developpement.
I would suggest "install-server-dev".

- make the contrib regressions work with pgxs. I'm not sure about what
is needed for that.

I'm available to fix any problem with this patch.

Have a nice day,

--
Fabien Coelho - coelho@cri.ensmp.fr

Content-Description:

[ Attachment, skipping... ]

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Where are we on this?

I think we're waiting on Peter to review it.

regards, tom lane

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Where are we on this?

I think we're waiting on Peter to review it.

Yes, indeed.

I've made two different submissions on this issue. It is up to the
reviewer/committer to chose, and I'm willing to solve any issue with these
patches if needed.

--
Fabien Coelho - coelho@cri.ensmp.fr

Well, we don't have forever to wait on this, and I don't plan to require
patches to be held until one specific person can review them. Fabien,
which version is your best? I will put it in the patch queue.

---------------------------------------------------------------------------

Fabien COELHO wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Where are we on this?

I think we're waiting on Peter to review it.

Yes, indeed.

I've made two different submissions on this issue. It is up to the
reviewer/committer to chose, and I'm willing to solve any issue with these
patches if needed.

--
Fabien Coelho - coelho@cri.ensmp.fr

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Where are we on this?

I think we're waiting on Peter to review it.

I have no technical issue with the patch, except that as I have
previously mentioned, I don't particularly care for what it does.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Where are we on this?

I think we're waiting on Peter to review it.

I have no technical issue with the patch, except that as I have
previously mentioned, I don't particularly care for what it does.

Yea, I am sure it isn't 100% ideal but what other options to we have if
we want this type of functionality, which I think we have agreed as a
group we do?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Fabien COELHO wrote:

Thus it suggests that the only use for the two targets is whether someone
is interested in having a smaller footprint version...

The other reason why I sticked to it in the submission is to be upward
compatible with the previous state with 2 differents targets. I feel it is
a "political" decision to shift to a single target install, and I'm not
the one to take such a decision... I'm only really interested in having
server headers and pgxs stuff installed by default, so that extensions can
be reliably added on a default installation.

So, the question remains the same: is there a consensus to drop
install-all-headers/light-install targets for a simple and full "install"?

This is the 8.0 release --- let's remove those targets. Our logic has
usually been to remove confusing options and document their removal
rather than keep them around and continue confusing people forever.

Would you resubmit with adjustments for comments made?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Dear Bruce,

So, the question remains the same: is there a consensus to drop
install-all-headers/light-install targets for a simple and full "install"?

This is the 8.0 release --- let's remove those targets. Our logic has
usually been to remove confusing options and document their removal
rather than keep them around and continue confusing people forever.

Would you resubmit with adjustments for comments made?

What comments do you want me to adjust?

Do you mean I should add a paragraph is the documentation about
that now there is only one install target. I updated the documentation
on both patch submission, but if more is required I'll do what I ask,
just be more specific.

The v2 version is the one which has a single and simple "install" target
everywhere.

--
Fabien.

Fabien COELHO wrote:

Dear Bruce,

So, the question remains the same: is there a consensus to drop
install-all-headers/light-install targets for a simple and full "install"?

This is the 8.0 release --- let's remove those targets. Our logic has
usually been to remove confusing options and document their removal
rather than keep them around and continue confusing people forever.

Would you resubmit with adjustments for comments made?

What comments do you want me to adjust?

Do you mean I should add a paragraph is the documentation about
that now there is only one install target. I updated the documentation
on both patch submission, but if more is required I'll do what I ask,
just be more specific.

The v2 version is the one which has a single and simple "install" target
everywhere.

Ah, I see v2 now. It was a little farther down in my mailbox. It is
in the patch queue now.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073