ci: Build standalone INSTALL file

Hi,

The attached patch had a slight bug. Also turned out that the CI environment
didn't have pandoc installed. Fixed that.

Greetings,

Andres Freund

On 20 Dec 2023, at 15:15, Andres Freund <andres@anarazel.de> wrote:

The attached patch had a slight bug. Also turned out that the CI environment
didn't have pandoc installed. Fixed that.

LGTM.

+ time make -s -j${BUILD_JOBS} -C doc/src/sgml all INSTALL
unrelated pet peeve: "make -C doc/src/sgml all" doesn't build all docs targets..

--
Daniel Gustafsson

Andres Freund <andres@anarazel.de> writes:

We fairly regularly have commits breaking the generation of INSTALL. IIRC we
recently discussed building it locally unconditionally, but I couldn't
immediately find that discussion. Until then, I think we should at least
build it in CI so that cfbot can warn.

I thought the plan was to get rid of that file, in pursuit of making
our distribution tarballs be more or less pure git pulls. Instead of
expending more effort on it, why not just push that project forward?
(IIRC, what we intended to do instead was to modify the top-level
README to point at the HTML install directions on the web.)

regards, tom lane

On Wed, Dec 20, 2023 at 11:36:28AM -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

We fairly regularly have commits breaking the generation of INSTALL. IIRC we
recently discussed building it locally unconditionally, but I couldn't
immediately find that discussion. Until then, I think we should at least
build it in CI so that cfbot can warn.

I thought the plan was to get rid of that file, in pursuit of making
our distribution tarballs be more or less pure git pulls. Instead of
expending more effort on it, why not just push that project forward?
(IIRC, what we intended to do instead was to modify the top-level
README to point at the HTML install directions on the web.)

Hmm. It depends on if the next release should include it or not, but
let me add my +1 for replacing it with a simple redirect.
--
Michael

On Wed, Dec 20, 2023 at 03:28:56PM +0100, Daniel Gustafsson wrote:

+ time make -s -j${BUILD_JOBS} -C doc/src/sgml all INSTALL
unrelated pet peeve: "make -C doc/src/sgml all" doesn't build all docs targets..

That seems relevant in terms of coverage. Why not just moving the
INSTALL bit to a different line?
--
Michael

On 2023-12-21 08:44:33 +0900, Michael Paquier wrote:

On Wed, Dec 20, 2023 at 03:28:56PM +0100, Daniel Gustafsson wrote:

+ time make -s -j${BUILD_JOBS} -C doc/src/sgml all INSTALL
unrelated pet peeve: "make -C doc/src/sgml all" doesn't build all docs targets..

That seems relevant in terms of coverage. Why not just moving the
INSTALL bit to a different line?

I am confused - which coverage could we be loosing here?

Hi,

On 2023-12-20 15:28:56 +0100, Daniel Gustafsson wrote:

+ time make -s -j${BUILD_JOBS} -C doc/src/sgml all INSTALL
unrelated pet peeve: "make -C doc/src/sgml all" doesn't build all docs targets..

Well, building the PDF takes a *long* time and is rarely required. I think
there's an argument for adding INSTALL to all - however, there's a reason not
to as well: It has pandoc as an additional dependency, which isn't small...

Andres

Hi,

On 2023-12-21 08:39:26 +0900, Michael Paquier wrote:

On Wed, Dec 20, 2023 at 11:36:28AM -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

We fairly regularly have commits breaking the generation of INSTALL. IIRC we
recently discussed building it locally unconditionally, but I couldn't
immediately find that discussion. Until then, I think we should at least
build it in CI so that cfbot can warn.

I thought the plan was to get rid of that file, in pursuit of making
our distribution tarballs be more or less pure git pulls. Instead of
expending more effort on it, why not just push that project forward?
(IIRC, what we intended to do instead was to modify the top-level
README to point at the HTML install directions on the web.)

Ah, right. I don't really care what solution we go for, just that as long as
we have INSTALL, we should make sure we don't regularly break it... Both
Michael and I have in the last couple weeks.

Hmm. It depends on if the next release should include it or not, but
let me add my +1 for replacing it with a simple redirect.

Are you going to submit a patch for that bit?

Greetings,

Andres Freund

On 21 Dec 2023, at 10:16, Andres Freund <andres@anarazel.de> wrote:

Hi,

On 2023-12-20 15:28:56 +0100, Daniel Gustafsson wrote:

+ time make -s -j${BUILD_JOBS} -C doc/src/sgml all INSTALL
unrelated pet peeve: "make -C doc/src/sgml all" doesn't build all docs targets..

Well, building the PDF takes a *long* time and is rarely required. I think
there's an argument for adding INSTALL to all - however, there's a reason not
to as well: It has pandoc as an additional dependency, which isn't small...

Yeah, I'm not advocating changing anything.

--
Daniel Gustafsson

Andres Freund <andres@anarazel.de> writes:

On 2023-12-21 08:39:26 +0900, Michael Paquier wrote:

On Wed, Dec 20, 2023 at 11:36:28AM -0500, Tom Lane wrote:

I thought the plan was to get rid of that file, in pursuit of making
our distribution tarballs be more or less pure git pulls. Instead of
expending more effort on it, why not just push that project forward?

Ah, right. I don't really care what solution we go for, just that as long as
we have INSTALL, we should make sure we don't regularly break it... Both
Michael and I have in the last couple weeks.

So let's just do it. I think the only real question is what URL
to point at exactly. We can't simply say

https://www.postgresql.org/docs/current/installation.html

because that will be wrong for any version more than one major
release back. We could make it version-specific,

https://www.postgresql.org/docs/17/installation.html

and task src/tools/version_stamp.pl with updating it. But that's
problematic for not-yet-released branches (there's no 17 today
for example). Perhaps we can use /devel/ in the master branch
and try to remember to replace that with a version number as soon
as a release branch is forked off --- but does the docs website
get populated as soon as the branch is made?

regards, tom lane

Hi,

On 2023-12-21 10:22:49 -0500, Tom Lane wrote:

I think the only real question is what URL to point at exactly. We can't
simply say

https://www.postgresql.org/docs/current/installation.html

because that will be wrong for any version more than one major
release back.

Right.

We could make it version-specific,

https://www.postgresql.org/docs/17/installation.html

and task src/tools/version_stamp.pl with updating it. But that's
problematic for not-yet-released branches (there's no 17 today
for example).

Perhaps we could make the website redirect 17 to /devel/ until 17 is branched
off?

Perhaps we can use /devel/ in the master branch
and try to remember to replace that with a version number as soon
as a release branch is forked off --- but does the docs website
get populated as soon as the branch is made?

I think it runs a few times a day - breaking the link for a few hours wouldn't
be optimal, but also not the end of the world. But redirecting $vnext -> to
devel would probably be a more reliable approach.

Greetings,

Andres Freund

Andres Freund <andres@anarazel.de> writes:

On 2023-12-21 10:22:49 -0500, Tom Lane wrote:

We could make it version-specific,
https://www.postgresql.org/docs/17/installation.html
and task src/tools/version_stamp.pl with updating it. But that's
problematic for not-yet-released branches (there's no 17 today
for example).

Perhaps we could make the website redirect 17 to /devel/ until 17 is branched
off?

Hmm, maybe, but then there's a moving part in version-stamping that's not
accessible to the average committer. On the other hand, it wouldn't
be too awful if that redirect didn't get updated instantly after a
branch. This is probably a point where we need advice from the web
team about how they manage documentation branches on the site.

Perhaps we can use /devel/ in the master branch
and try to remember to replace that with a version number as soon
as a release branch is forked off --- but does the docs website
get populated as soon as the branch is made?

I think it runs a few times a day - breaking the link for a few hours wouldn't
be optimal, but also not the end of the world. But redirecting $vnext -> to
devel would probably be a more reliable approach.

Let's go with "/devel/ in master and a number in release branches"
for now, and tweak that if the web team wants to take on maintaining
a redirect. I'll put together a concrete patch proposal in a little
bit.

regards, tom lane

Hi,

On 2023-12-21 10:46:02 -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2023-12-21 10:22:49 -0500, Tom Lane wrote:

We could make it version-specific,
https://www.postgresql.org/docs/17/installation.html
and task src/tools/version_stamp.pl with updating it. But that's
problematic for not-yet-released branches (there's no 17 today
for example).

Perhaps we could make the website redirect 17 to /devel/ until 17 is branched
off?

Hmm, maybe, but then there's a moving part in version-stamping that's not
accessible to the average committer. On the other hand, it wouldn't
be too awful if that redirect didn't get updated instantly after a
branch. This is probably a point where we need advice from the web
team about how they manage documentation branches on the site.

IIRC the relevant part of the website code has access to a table of
documentation versions, so the redirect could be implement based on not
knowing the version.

Perhaps we can use /devel/ in the master branch
and try to remember to replace that with a version number as soon
as a release branch is forked off --- but does the docs website
get populated as soon as the branch is made?

I think it runs a few times a day - breaking the link for a few hours wouldn't
be optimal, but also not the end of the world. But redirecting $vnext -> to
devel would probably be a more reliable approach.

Let's go with "/devel/ in master and a number in release branches"
for now, and tweak that if the web team wants to take on maintaining
a redirect. I'll put together a concrete patch proposal in a little
bit.

Cool.

Greetings,

Andres Freund

Andres Freund <andres@anarazel.de> writes:

On 2023-12-21 10:46:02 -0500, Tom Lane wrote:

Let's go with "/devel/ in master and a number in release branches"
for now, and tweak that if the web team wants to take on maintaining
a redirect. I'll put together a concrete patch proposal in a little
bit.

Cool.

Here's a draft patch for this. Most of it is mechanical removal of
infrastructure for building the INSTALL file. If anyone wants to
bikeshed on the new wording of README, feel free.

regards, tom lane

On Thu, Dec 21, 2023 at 02:22:02PM -0500, Tom Lane wrote:

Here's a draft patch for this. Most of it is mechanical removal of
infrastructure for building the INSTALL file. If anyone wants to
bikeshed on the new wording of README, feel free.

Thanks for putting this together. That looks reasonable.

diff --git a/README b/README
index 56d0c951a9..e40e610ccb 100644
--- a/README
+++ b/README
@@ -9,14 +9,13 @@ that supports an extended subset of the SQL standard, including
-See the file INSTALL for instructions on how to build and install
-PostgreSQL.  That file also lists supported operating systems and
-hardware platforms and contains information regarding any other
-software packages that are required to build or run the PostgreSQL
-system.  Copyright and license information can be found in the
-file COPYRIGHT.  A comprehensive documentation set is included in this
-distribution; it can be read as described in the installation
-instructions.
+Copyright and license information can be found in the file COPYRIGHT.
+
+General documentation about this version of PostgreSQL can be found at:
+https://www.postgresql.org/docs/devel/
+In particular, information about building PostgreSQL from the source
+code can be found at:
+https://www.postgresql.org/docs/devel/installation.html

Sounds fine by me, including the extra step documented in
RELEASE_CHANGES. No information is lost.
--
Michael

Michael Paquier <michael@paquier.xyz> writes:

On Thu, Dec 21, 2023 at 02:22:02PM -0500, Tom Lane wrote:

Here's a draft patch for this. Most of it is mechanical removal of
infrastructure for building the INSTALL file. If anyone wants to
bikeshed on the new wording of README, feel free.

Thanks for putting this together. That looks reasonable.

Thanks for checking it. Pushed --- we can tweak things later
if we decide the web-redirect idea is superior to this.

regards, tom lane