WIP: About CMake v2

YUriy Zhuravlev wrote:

Hello Hackers

How would you react if I provided a patch which introduces a CMake build
system?

What's your motivation for doing so?

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-08-28 12:32:45 -0300, Alvaro Herrera wrote:

YUriy Zhuravlev wrote:

Hello Hackers

How would you react if I provided a patch which introduces a CMake build
system?

What's your motivation for doing so?

I definitely can see some advantages. Non-broken dependencies around
recursive make being a major one. But I'm also afraid it's a rather
large undertaking. There's a fair number of special kind of rules, and
we're probably not going to want to break pgxs for extensions.

I also have some doubts around the portability of cmake and it's
generated makefiles. We do support some odd platforms.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund wrote:

On 2015-08-28 12:32:45 -0300, Alvaro Herrera wrote:

YUriy Zhuravlev wrote:

Hello Hackers

How would you react if I provided a patch which introduces a CMake build
system?

What's your motivation for doing so?

I definitely can see some advantages. Non-broken dependencies around
recursive make being a major one. But I'm also afraid it's a rather
large undertaking. There's a fair number of special kind of rules, and
we're probably not going to want to break pgxs for extensions.

I also have some doubts around the portability of cmake and it's
generated makefiles. We do support some odd platforms.

If it allows us to get rid of our custom MSVC scripts, it's a huge
benefit, for sure -- that has been a huge pain in the neck since day
one.

Like you, I am also afraid it's a huge undertaking, but if Uri wants to
tackle it, we have that part covered. Experimentation until we get it
all correct is going to waste some of everybody's time, too, I'm sure.

I wonder about two other things: one is speed of the build (not that
currently it's all that great, given all the mess with recursive make
invocations, but perhaps it can be even worse); the other is how ugly
the generated files are going to be, and are we going to carry them in
our repo -- right now we only have configure, but are we going to keep
extra files to cope with builds in systems that don't have cmake
installed (as we cope with missing bison and flex)?

I sure would oppose something that looks as ugly as Automake makefiles.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-08-28 13:28:49 -0300, Alvaro Herrera wrote:

the other is how ugly the generated files are going to be, and are we
going to carry them in our repo -- right now we only have configure,
but are we going to keep extra files to cope with builds in systems
that don't have cmake installed (as we cope with missing bison and
flex)?

Apparently it's still unsupported to ship generated files - they contain
absolute paths and such. Personally I think that might be the
death-knell - I doubt that we want to have a full dependency on cmake on
every platform?

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

I wonder about two other things: one is speed of the build (not that
currently it's all that great, given all the mess with recursive make
invocations, but perhaps it can be even worse); the other is how ugly
the generated files are going to be, and are we going to carry them in
our repo -- right now we only have configure, but are we going to keep
extra files to cope with builds in systems that don't have cmake
installed (as we cope with missing bison and flex)?

As near as I can tell, the generated files are platform-specific.
(They're certainly different for Unix and Windows; the overview
I'm looking at doesn't say in so many words whether they can vary
at a finer grain, but I bet they do.) So I'm afraid cmake would
likely become a build requirement, even for tarball users. That
is probably not a show-stopper, but it's a point against the idea.

I have no idea whether switching to cmake would be a good thing or not.
It's possible that it'd end up being even uglier than our current
autoconf+gmake+msvc-scripts mess ... although when phrased that way,
that sounds like a pretty low bar to clear. Anyway, if YUriy is willing
to do the preliminary investigation, let's see what he comes up with.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

It's broadly interesting, but since it bakes in a build dependency on
CMake, there is some risk that the dependencies become an insurmountable
problem.

(Does CMake run on a VAX 11/780?? :-))

It is probably worth a try, to see what improvements arise, albeit with the
need to accept some risk of refusal of the change.

The experiment is most likely necessary: we won't know the benefits without
trying.

If the results represent little improvement, there will be little or no
appetite to jump through the dependency hoops needed to get the change
accepted.

On the other hand, if there are big gains, that encourages pushing thru the
dependency issues.
On Aug 28, 2015 10:45, "YUriy Zhuravlev" <u.zhuravlev@postgrespro.ru> wrote:

Christopher Browne <cbbrowne@gmail.com> writes:

(Does CMake run on a VAX 11/780?? :-))

Yeah. I see the two major risks as being:

1. We limit ourselves to platforms that cmake works on.

2. We lose the ability to handle weird special-case tests that are
possible (if not necessarily pleasant) with autoconf/gmake.

I might be overly worried about #2 --- but the impression I have of cmake
is that they've spent lots of time on "make easy cases easy" and maybe not
enough on "make hard cases possible".

Anyway, we won't know unless somebody tries it. There will certainly be
some pluses and some minuses, and we'll have to decide whether the pluses
outweigh the minuses. I don't think we can tell that without a fairly
detailed attempt at making it work.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Friday 28 August 2015 13:28:49 Alvaro Herrera wrote:

Andres Freund wrote:

On 2015-08-28 12:32:45 -0300, Alvaro Herrera wrote:

YUriy Zhuravlev wrote:

Hello Hackers

How would you react if I provided a patch which introduces a CMake
build
system?

What's your motivation for doing so?

I definitely can see some advantages. Non-broken dependencies around
recursive make being a major one. But I'm also afraid it's a rather
large undertaking. There's a fair number of special kind of rules, and
we're probably not going to want to break pgxs for extensions.

I also have some doubts around the portability of cmake and it's
generated makefiles. We do support some odd platforms.

If it allows us to get rid of our custom MSVC scripts, it's a huge
benefit, for sure -- that has been a huge pain in the neck since day
one.

As a user, I suffer a bit with the current build system when I try to build
libpq in MinGW. So, if moving to CMake means the build is more 'decoupled',
i.e. less inter-dependent modules, it would be great.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Friday 28 August 2015 13:51:30 you wrote:

It's broadly interesting, but since it bakes in a build dependency on
CMake, there is some risk that the dependencies become an insurmountable
problem.

(Does CMake run on a VAX 11/780?? :-))

http://public.kitware.com/Bug/view.php?id=13605 you about this?

It is probably worth a try, to see what improvements arise, albeit with the
need to accept some risk of refusal of the change.

The experiment is most likely necessary: we won't know the benefits without
trying.

You right.

If the results represent little improvement, there will be little or no
appetite to jump through the dependency hoops needed to get the change
accepted.

On the other hand, if there are big gains, that encourages pushing thru the
dependency issues.

On Aug 28, 2015 10:45, "YUriy Zhuravlev" <u.zhuravlev@postgrespro.ru> wrote:

Hello Hackers

How would you react if I provided a patch which introduces a CMake build
system?

Old thread:
/messages/by-id/200812291325.13354.peter_e@gmx.net

The main argument against the "it's too hard". I'm right?

Thanks!
--
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

--
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

YUriy Zhuravlev wrote:

On Friday 28 August 2015 13:51:30 you wrote:

It's broadly interesting, but since it bakes in a build dependency on
CMake, there is some risk that the dependencies become an insurmountable
problem.

(Does CMake run on a VAX 11/780?? :-))

http://public.kitware.com/Bug/view.php?id=13605 you about this?

Hmm, what this closed bug report says to me is that 1) cmake developers
take portability seriously, 2) all common and even some not-so-common
platforms are likely to be in hand already. Therefore, either we will
not find any portability problems, or fixing upstream those we do find
will not be terribly difficult.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-08-29 17:53:26 -0300, Alvaro Herrera wrote:

Therefore, either we will not find any portability problems, or fixing
upstream those we do find will not be terribly difficult.

Well, the difference to know is that we can't resolve that relatively
quickly ourselves, but that it'd rather require $whoever installing a
new version of cmake...

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund <andres@anarazel.de> writes:

On 2015-08-29 17:53:26 -0300, Alvaro Herrera wrote:

Therefore, either we will not find any portability problems, or fixing
upstream those we do find will not be terribly difficult.

Well, the difference to know is that we can't resolve that relatively
quickly ourselves, but that it'd rather require $whoever installing a
new version of cmake...

Yup. So one of the things we'd want to establish is what's the minimum
version of cmake we're going to require, and which (hopefully old)
platforms will we be leaving out in the cold with that?

I suspect it's premature to try to decide that now, but it's certainly
a question to keep in mind.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2015-08-29 17:53:26 -0300, Alvaro Herrera wrote:

Therefore, either we will not find any portability problems, or fixing
upstream those we do find will not be terribly difficult.

Well, the difference to know is that we can't resolve that relatively
quickly ourselves, but that it'd rather require $whoever installing a
new version of cmake...

Yup. So one of the things we'd want to establish is what's the minimum
version of cmake we're going to require, and which (hopefully old)
platforms will we be leaving out in the cold with that?

Just for the record, in Debian land the versions shipped are thus:

oldoldstable (squeeze): 2.8.2 (has 2.8.9 available in squeeze-backports)
oldstable (wheezy): 2.8.9 (has 2.8.11 available in wheezy-backports)
stable (jessie): 3.0.2

VAX support was added in 2.8.11 (bug fixed in Oct. 2012).

I imagine that esoteric platforms are not going to have cmake at all and
are going to need their own installation anyway. Not sure if that's
going to be more onerous than the requirement to install GNU make.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 8/28/15 10:14 AM, YUriy Zhuravlev wrote:

Hello Hackers

How would you react if I provided a patch which introduces a CMake build
system?

I would say, "Yay!".

I had that on my todo list, but I won't mind if someone else does it.
I've certainly "had it" with the current build system. ;-)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

I imagine that esoteric platforms are not going to have cmake at all and
are going to need their own installation anyway. Not sure if that's
going to be more onerous than the requirement to install GNU make.

If we get to the point where this is starting to look like a real
proposal, I'll try to install cmake from source on my buildfarm critters,
none of which have it ATM. And we'll need to get other buildfarm owners
to do likewise. So we'll be able to acquire data points on whether that's
a big problem or not.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Thanks all hackers.
I have not heard of fundamental problems and continue its development. :)

--
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Aug 28, 2015 at 01:28:49PM -0300, Alvaro Herrera wrote:

If it allows us to get rid of our custom MSVC scripts, it's a huge
benefit, for sure -- that has been a huge pain in the neck since day
one.

Moreover, I suggest beginning with a patch that replaces the src/tools/msvc
build system with CMake. One can test almost all Windows configurations using
two machines, so author and reviewer can test most everything. src/tools/msvc
has more defects than the GNU make build system, so the minimum-quality
acceptable replacement is easier to deliver. With CMake driving all MSVC
builds, it would be much easier to gradually extend to other platforms.

Thanks,
nm

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Noah Misch <noah@leadboat.com> writes:

On Fri, Aug 28, 2015 at 01:28:49PM -0300, Alvaro Herrera wrote:

If it allows us to get rid of our custom MSVC scripts, it's a huge
benefit, for sure -- that has been a huge pain in the neck since day
one.

Moreover, I suggest beginning with a patch that replaces the src/tools/msvc
build system with CMake. One can test almost all Windows configurations using
two machines, so author and reviewer can test most everything. src/tools/msvc
has more defects than the GNU make build system, so the minimum-quality
acceptable replacement is easier to deliver. With CMake driving all MSVC
builds, it would be much easier to gradually extend to other platforms.

mmm ... dunno about that plan. In the first place, you seem to be
recommending that YUriy incorporate all the deficiencies of src/tools/msvc
into his first draft and then hope somehow to fix them later. In the
second place, I think that the core problem here involves being sure
we can still cover all the portability issues that have been laboriously
hacked into the autoconf/gmake build system over nigh two decades.
Starting from a build system that was designed to cover exactly one target
platform, and has never been asked to do anything more than that, doesn't
sound like a recipe for success there.

Maybe you're right about how to approach this, but I wouldn't bet on it.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Aug 28, 2015 at 11:39 AM, Andres Freund <andres@anarazel.de> wrote:

I definitely can see some advantages. Non-broken dependencies around
recursive make being a major one. But I'm also afraid it's a rather
large undertaking. There's a fair number of special kind of rules, and
we're probably not going to want to break pgxs for extensions.

Maybe we should merge all of the makefiles for subdirectories of
src/backend into a single makefile. The major disadvantage would be
that you couldn't rebuild a subdirectory any more by typing make -C
src/backend/executor or whatever. And I do do that sometimes, so
maybe it would be annoying, but presumably it would make the
dependency issues a lot easier to deal with.

I guess I'm a bit skeptical about the idea of porting to a new build
system. There's a good chance of replacing the problems we know about
with new problems that are no less serious, but merely unknown to us.
But I'm not going to stand here and hold my breath if everyone else
thinks it's a good idea.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers