HEAD build failure on win32 mingw

On Thu, 2008-11-20 at 08:51 +0900, ITAGAKI Takahiro wrote:

HEAD is failed to be built on win32 mingw.
It requires manual 'mkdir man7' or so.

Are there any changes in build process?

It is probably because of this commit:

http://archives.postgresql.org/pgsql-committers/2008-11/msg00169.php

--
Devrim GÜNDÜZ, RHCE
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org

Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= <devrim@gunduz.org> writes:

On Thu, 2008-11-20 at 08:51 +0900, ITAGAKI Takahiro wrote:

HEAD is failed to be built on win32 mingw.
It requires manual 'mkdir man7' or so.

Are there any changes in build process?

It is probably because of this commit:
http://archives.postgresql.org/pgsql-committers/2008-11/msg00169.php

Since all the buildfarm machines are still green, a report that "HEAD is
broken on mingw" is entirely useless. Takahiro-san needs to expend some
effort on determining why his machine is yielding different results from
those ones.

regards, tom lane

ITAGAKI Takahiro wrote:

HEAD is failed to be built on win32 mingw.
It requires manual 'mkdir man7' or so.

Are there any changes in build process?
Or am I missing something required to build?

----
$ make
/bin/sh.exe: man7/.timestamp: No such file or directory

$ mkdir doc/man7 <- HERE
$ make
(succeeded to build)

$ make install
../config/install-sh: man7/*.7 does not exist.

$ touch doc/man7/dummy.7 <- HERE
$ make install
(succeeded to install)

----

This code would only be executed if you have a man.tar.gz in the doc
directory. If you do, it is probably an old one that indeed does not
contain the man7 directory. So delete the man.tar.gz (and build a new
one if you are so inclined).

Peter Eisentraut <peter_e@gmx.net> wrote:

This code would only be executed if you have a man.tar.gz in the doc
directory. If you do, it is probably an old one that indeed does not
contain the man7 directory. So delete the man.tar.gz (and build a new
one if you are so inclined).

I used a nightly snapshot(snapshot/postgresql-snapshot.tar.bz2)
and it includes *.gz files. It would be the cause.

Thanks. I'll delete the file before building.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center

On Thu, Nov 20, 2008 at 4:31 AM, ITAGAKI Takahiro
<itagaki.takahiro@oss.ntt.co.jp> wrote:

I used a nightly snapshot(snapshot/postgresql-snapshot.tar.bz2)
and it includes *.gz files. It would be the cause.

i saw the same on a nightly snapshot a week ago, then used cvs and
everything was good... i thouhgt it was a problem already solved...
could be a problem in the nightly builds?

--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

Jaime Casanova wrote:

On Thu, Nov 20, 2008 at 4:31 AM, ITAGAKI Takahiro
<itagaki.takahiro@oss.ntt.co.jp> wrote:

I used a nightly snapshot(snapshot/postgresql-snapshot.tar.bz2)
and it includes *.gz files. It would be the cause.

i saw the same on a nightly snapshot a week ago, then used cvs and
everything was good... i thouhgt it was a problem already solved...
could be a problem in the nightly builds?

Yeah. Buildfarm doesn't do docs at all.

Looks to me at first glance like this line in the Makefile:

$(mkinstalldirs) man$(sqlmansectnum)

should be moved up so it comes immediately after gzip line. That
certainly seems to make it work properly for me.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

Jaime Casanova wrote:

i saw the same on a nightly snapshot a week ago, then used cvs and
everything was good... i thouhgt it was a problem already solved...
could be a problem in the nightly builds?

Yeah. Buildfarm doesn't do docs at all.

Looks to me at first glance like this line in the Makefile:
$(mkinstalldirs) man$(sqlmansectnum)
should be moved up so it comes immediately after gzip line.

No, that is just a kluge that prevents the visible failure.

It looks to me like the problem is that the makefile expects the
man.tar.gz file to contain stuff in man1/ and man7/, but the file
that is actually being delivered in snapshots still contains man1/
and manl/. So that file needs to be regenerated. Peter?

regards, tom lane

Tom Lane wrote:

It looks to me like the problem is that the makefile expects the
man.tar.gz file to contain stuff in man1/ and man7/, but the file
that is actually being delivered in snapshots still contains man1/
and manl/. So that file needs to be regenerated. Peter?

Shouldn't it be part of the nightly snapshot creation to make that file?
In the snapshot I just downloaded its date is yesterday.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

Tom Lane wrote:

... the file
that is actually being delivered in snapshots still contains man1/
and manl/. So that file needs to be regenerated. Peter?

Shouldn't it be part of the nightly snapshot creation to make that file?
In the snapshot I just downloaded its date is yesterday.

The tar file's own date might be yesterday, but its contents are dated
2003-11-02 ... and indeed look to be that old. So something's bollixed
in the snapshot generation process.

Historically the man.tar.gz files were created manually because there
were some manual fixups needed to the generated man files. I'm not sure
what vestiges of that still remain --- Peter's generally been the one to
take care of it. But we definitely aren't shipping a freshly generated
copy in the nightly snapshot right now.

If we do have a fully automated process now, it's probably fair to ask
why there's an internal tarball involved at all, rather than just
shipping the built man1/ and man7/ subdirectories (and likewise for the
html). Double compression of that data isn't going to be helpful.

regards, tom lane

On Friday 21 November 2008 23:33:47 Tom Lane wrote:

Historically the man.tar.gz files were created manually because there
were some manual fixups needed to the generated man files. I'm not sure
what vestiges of that still remain --- Peter's generally been the one to
take care of it. But we definitely aren't shipping a freshly generated
copy in the nightly snapshot right now.

I don't actually know how the snapshots are built, so I don't know how to fix
it.

If we do have a fully automated process now, it's probably fair to ask
why there's an internal tarball involved at all, rather than just
shipping the built man1/ and man7/ subdirectories (and likewise for the
html). Double compression of that data isn't going to be helpful.

Fixing all this is definitely my plan for 8.4, but we are not there yet.

Peter Eisentraut wrote:

On Friday 21 November 2008 23:33:47 Tom Lane wrote:

Historically the man.tar.gz files were created manually because there
were some manual fixups needed to the generated man files. I'm not sure
what vestiges of that still remain --- Peter's generally been the one to
take care of it. But we definitely aren't shipping a freshly generated
copy in the nightly snapshot right now.

I don't actually know how the snapshots are built, so I don't know how to fix
it.

According to Stefan Kaltenbrunner, who took the time to delve into
developer.pg.org guts, the script used to generate the snapshots looks
like this

/usr/bin/cvs -q export -rREL8_3_STABLE pgsql
cd pgsql
./configure
cd doc/src
gmake postgres.tar.gz
mv postgres.tar.gz ..
gmake postgres.tar.gz
mv postgres.tar.gz ..
gmake man.tar.gz
mv man.tar.gz ..
cd sgml
gmake HISTORY INSTALL
mv -f HISTORY INSTALL ../../..
cd ../../..
gmake VERSION=snapshot dist
gmake maintainer-clean

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera <alvherre@commandprompt.com> writes:

According to Stefan Kaltenbrunner, who took the time to delve into
developer.pg.org guts, the script used to generate the snapshots looks
like this

/usr/bin/cvs -q export -rREL8_3_STABLE pgsql
cd pgsql
./configure
cd doc/src
gmake postgres.tar.gz
mv postgres.tar.gz ..
gmake postgres.tar.gz
mv postgres.tar.gz ..
gmake man.tar.gz
mv man.tar.gz ..
cd sgml
gmake HISTORY INSTALL
mv -f HISTORY INSTALL ../../..
cd ../../..
gmake VERSION=snapshot dist
gmake maintainer-clean

That can't be the right script --- the final "maintainer-clean" would
wipe out all the derived files made by the dist step. Also, it's
*definitely* not the script being used to generate HEAD-branch
snapshots ...

regards, tom lane

Tom Lane wrote:

Alvaro Herrera <alvherre@commandprompt.com> writes:

According to Stefan Kaltenbrunner, who took the time to delve into
developer.pg.org guts, the script used to generate the snapshots looks
like this

/usr/bin/cvs -q export -rREL8_3_STABLE pgsql
cd pgsql
./configure
cd doc/src
gmake postgres.tar.gz
mv postgres.tar.gz ..
gmake postgres.tar.gz
mv postgres.tar.gz ..
gmake man.tar.gz
mv man.tar.gz ..
cd sgml
gmake HISTORY INSTALL
mv -f HISTORY INSTALL ../../..
cd ../../..
gmake VERSION=snapshot dist
gmake maintainer-clean

That can't be the right script --- the final "maintainer-clean" would
wipe out all the derived files made by the dist step. Also, it's
*definitely* not the script being used to generate HEAD-branch
snapshots ...

no it is not the -HEAD situation is significantly worse because what we
are actually shipping as the manpages there are the ones from 7.4 ...

set -e
export PATH=/usr/local/bin:/usr/bin:/bin:/sbin
export CVSROOT=:ext:scrappy@cvs.postgresql.org:/cvsroot
cd /usr/local/pgsql/snapshot
if [ -d pgsql ]
then
mv pgsql pgsql.dead
fi
if [ -d pgsql.dead ]
then
rm -rf pgsql.dead &
fi
/usr/bin/cvs -q export -rHEAD pgsql
cd pgsql
./configure
cd doc/src
gmake postgres.tar.gz
mv postgres.tar.gz ..
cp ~ftp/pub/dev/doc/man-7.4.tar.gz ../man.tar.gz
cd sgml
gmake HISTORY INSTALL
mv -f HISTORY INSTALL ../../..
cd ../../..
gmake VERSION=snapshot dist
gmake maintainer-clean

for x in *.tar.gz; do
md5 $x > $x.md5
done

cp *.tar.gz *.tar.gz.md5 /var/spool/ftp/pub/snapshot/dev

gunzip *.tar.gz
bzip2 -f *.tar

for x in *.tar.bz2; do
md5 $x > $x.md5
done

mv *.tar.bz2 *.tar.bz2.md5 /var/spool/ftp/pub/snapshot/dev

Stefan

Tom Lane wrote:

Alvaro Herrera <alvherre@commandprompt.com> writes:

gmake VERSION=snapshot dist
gmake maintainer-clean

That can't be the right script --- the final "maintainer-clean" would
wipe out all the derived files made by the dist step. Also, it's
*definitely* not the script being used to generate HEAD-branch
snapshots ...

Stefan already posted the correct one, but to clarify why it runs with
the same "make dist ; make maintainer-clean", what's happening is that
dist creates a tarball, so by the time maintainer-clean runs, the files
have already been packed.

The derived files are created by distprep AFAIK.

BTW most of that script is just duplicating the stuff done in distdir ...

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Stefan Kaltenbrunner wrote:

no it is not the -HEAD situation is significantly worse because what we
are actually shipping as the manpages there are the ones from 7.4 ...

So how do we get Marc to fix this?

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera <alvherre@commandprompt.com> writes:

So how do we get Marc to fix this?

Actually I think the ball is in Peter's court now. Marc updated the
snapshot build script, but all the attempts are dying now because the
new docbook2man part of the Makefiles doesn't work on svr1.

regards, tom lane

On Friday 28 November 2008 18:25:23 Tom Lane wrote:

Alvaro Herrera <alvherre@commandprompt.com> writes:

So how do we get Marc to fix this?

Actually I think the ball is in Peter's court now. Marc updated the
snapshot build script, but all the attempts are dying now because the
new docbook2man part of the Makefiles doesn't work on svr1.

As far as I know, the snapshot and release building scripts have always taken
the existing man.tar.gz from the FTP server and put it into the release
tarball. This is why the mismatch appeared in the first place, after all. I
have uploaded an updated and matching man.tar.gz a while ago already, so
things should be working again.

This has nothing to do with the new man page build stuff. Anything official
should still be using the old stuff. And the snapshot build shouldn't be
building anything either way.

Peter Eisentraut <peter_e@gmx.net> writes:

On Friday 28 November 2008 18:25:23 Tom Lane wrote:

Actually I think the ball is in Peter's court now. Marc updated the
snapshot build script, but all the attempts are dying now because the
new docbook2man part of the Makefiles doesn't work on svr1.

As far as I know, the snapshot and release building scripts have always taken
the existing man.tar.gz from the FTP server and put it into the release
tarball. This is why the mismatch appeared in the first place, after all. I
have uploaded an updated and matching man.tar.gz a while ago already, so
things should be working again.

This has nothing to do with the new man page build stuff. Anything official
should still be using the old stuff. And the snapshot build shouldn't be
building anything either way.

Are you sure you're not remembering How Things Used To Be?

AFAICT, we are building both man.tar.gz and postgres.tar.gz on-the-fly
in current releases. At least, both of them have internal timestamps
supporting that theory in the official 8.3.5 and 8.2.11 tarballs, and
the contained files reflect recent source changes. It looks to me like
the dev snapshot build process has failed to track the official releases.

regards, tom lane

On Saturday 29 November 2008 00:52:03 Tom Lane wrote:

AFAICT, we are building both man.tar.gz and postgres.tar.gz on-the-fly
in current releases. At least, both of them have internal timestamps
supporting that theory in the official 8.3.5 and 8.2.11 tarballs, and
the contained files reflect recent source changes. It looks to me like
the dev snapshot build process has failed to track the official releases.

Well, Stefan showed upthread that the snapshot script contains this line:

cp ~ftp/pub/dev/doc/man-7.4.tar.gz ../man.tar.gz

This needs to be changed to copy man.tar.gz instead.

Releases might work differently, but that is not really the problem at hand.