build farm machine using <make -j 8> mixed results

Excerpts from Robert Creager's message of sáb sep 01 12:12:51 -0400 2012:

I change the build-farm.conf file to have the following make line:

make => 'make -j 8', # or gmake if required. can include path if necessary.

2 pass, 4 fail. Is this a build configuration you want to pursue?

Sure, why not? Parallel building is going to become more common, so
it's a good idea to investigate the failures. I would have it build
only HEAD though, because we're not likely to backpatch these fixes.

I can either create a new machine, or change one of my existing machines. Makes no difference to me.

If we want to have it run only HEAD I would say you should create a new
animal.

Perhaps you should wait until after 9.2 has been released, though, to
avoid distracting people :-)

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 9/1/12 12:12 PM, Robert Creager wrote:

I change the build-farm.conf file to have the following make line:

make => 'make -j 8', # or gmake if required. can include path if
necessary.

2 pass, 4 fail. Is this a build configuration you want to pursue?

Sure that would be useful, but it's pretty clear that the check stages
don't work in parallel. It think it's because the ports conflict, but
there could be any number of other problems.

That said, it would be useful, in my mind, to support parallel checks.
But unless someone is going to put in the work first, you should
restrict your parallel runs to the build and install phases.

Excerpts from Peter Eisentraut's message of mar sep 04 18:49:46 -0300 2012:

On 9/1/12 12:12 PM, Robert Creager wrote:

I change the build-farm.conf file to have the following make line:

make => 'make -j 8', # or gmake if required. can include path if
necessary.

2 pass, 4 fail. Is this a build configuration you want to pursue?

Sure that would be useful, but it's pretty clear that the check stages
don't work in parallel. It think it's because the ports conflict, but
there could be any number of other problems.

Is that really the problem? As far as I know, buildfarm doesn't use
anything like installcheck-world or similar targets; each check target
is run separately, serially, by the BF script.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 09/04/2012 05:49 PM, Peter Eisentraut wrote:

On 9/1/12 12:12 PM, Robert Creager wrote:

I change the build-farm.conf file to have the following make line:

make => 'make -j 8', # or gmake if required. can include path if
necessary.

2 pass, 4 fail. Is this a build configuration you want to pursue?

Sure that would be useful, but it's pretty clear that the check stages
don't work in parallel. It think it's because the ports conflict, but
there could be any number of other problems.

That said, it would be useful, in my mind, to support parallel checks.
But unless someone is going to put in the work first, you should
restrict your parallel runs to the build and install phases.

The buildfarm code doesn't contain a facility to use a different make
incantation for each step. It's pretty much an all or nothing deal. Of
course, you can hack run_build.pl to make it do that, but I highly
discourage that. For one thing, it makes upgrading that much more
difficult. All the tweaking is supposed to be done vie the config file.
I guess I could add a setting that allowed for per step make flags.

Frankly, I have had enough failures of parallel make that I think doing
this would generate a significant number of non-repeatable failures (I
had one just the other day that took three invocations of make to get
right). So I'm not sure doing this would advance us much, although I'm
open to persuasion.

cheers

andrew

On Sep 4, 2012 6:06 PM, "Andrew Dunstan" <andrew@dunslane.net> wrote:

Frankly, I have had enough failures of parallel make that I think doing

this would generate a significant number of non-repeatable failures (I had
one just the other day that took three invocations of make to get right).
So I'm not sure doing this would advance us much, although I'm open to
persuasion.

Seeing as most PostgreSQL bugs appear with concurrency, I think we should
leave our default config with 1 for max connections.

;-)

Parallel make failures are bugs in the dependencies as described in our
make files.

For the build phase, I don't recall parallel problems and as a habit I
usually use parallel makes. I would like that to be supported and I think
I've seen fixes applied when we had issues before. Cutting build times to
1/2 to 1/4 is good.

Checks and tests are harder because often they can't run in parallel. But
then we shouldn't have them listed as independent prerequisites for
targets. Ideally. But fixing it might not be worth the cost since an
"acceptable" work around (rely upon make to serialize the test sequences in
the particular order) is pretty painless (so far).

Of course, having the ability to run the tests 8 at a time (or more) and
reduce the time by 80% would be nice .;-)
On Sep 4, 2012 6:06 PM, "Andrew Dunstan" <andrew@dunslane.net> wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Frankly, I have had enough failures of parallel make that I think doing
this would generate a significant number of non-repeatable failures (I
had one just the other day that took three invocations of make to get
right). So I'm not sure doing this would advance us much, although I'm
open to persuasion.

Really? I routinely use -j4 for building, and it's been a long time
since I've seen failures. I can believe that for instance "make check"
in contrib would have a problem running in parallel, but the build
process per se seems reliable enough from here.

regards, tom lane

On 09/04/2012 08:37 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Frankly, I have had enough failures of parallel make that I think doing
this would generate a significant number of non-repeatable failures (I
had one just the other day that took three invocations of make to get
right). So I'm not sure doing this would advance us much, although I'm
open to persuasion.

Really? I routinely use -j4 for building, and it's been a long time
since I've seen failures. I can believe that for instance "make check"
in contrib would have a problem running in parallel, but the build
process per se seems reliable enough from here.

Both cases were vpath builds, which is what I usually use, if that's a
useful data point.

Maybe I run on lower level hardware than you do. I saw this again this
afternoon after I posted the above. In both cases this was the machine
that runs the buildfarm's crake. I'll try to get a handle on it.

cheers

andrew

On 09/04/2012 08:51 PM, Andrew Dunstan wrote:

On 09/04/2012 08:37 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Frankly, I have had enough failures of parallel make that I think doing
this would generate a significant number of non-repeatable failures (I
had one just the other day that took three invocations of make to get
right). So I'm not sure doing this would advance us much, although I'm
open to persuasion.

Really? I routinely use -j4 for building, and it's been a long time
since I've seen failures. I can believe that for instance "make check"
in contrib would have a problem running in parallel, but the build
process per se seems reliable enough from here.

Both cases were vpath builds, which is what I usually use, if that's a
useful data point.

Maybe I run on lower level hardware than you do. I saw this again this
afternoon after I posted the above. In both cases this was the machine
that runs the buildfarm's crake. I'll try to get a handle on it.

Well, it looks like it's always failing on ecpg, with preproc.h not
being made in the right order. Here is the last bit of a make log
starting from when it starts on ecpg. This is pretty repeatable.

cheers

andrew

-----

make -C ecpg all
make[3]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg'
make -C include all
make[4]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/include'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/include'
make -C pgtypeslib all
make -C preproc all
make[4]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
numeric.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/numeric.c -MMD
-MP -MF .deps/numeric.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
common.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/common.c -MMD
-MP -MF .deps/common.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
datetime.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/datetime.c -MMD
-MP -MF .deps/datetime.Po
make[4]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/preproc'
make -C ../../../../src/port all
make[5]: Entering directory `/home/pgl/npgl/vpath.testpar/src/port'
make -C ../backend submake-errcodes
make[6]: Entering directory `/home/pgl/npgl/vpath.testpar/src/backend'
make[6]: Nothing to be done for `submake-errcodes'.
make[6]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/backend'
make[5]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/port'
'/usr/bin/perl'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parse.pl
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc <
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/../../../backend/parser/gram.y

preproc.y

ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
dt_common.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/dt_common.c -MMD
-MP -MF .deps/dt_common.Po
'/usr/bin/perl'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/check_rules.pl
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/../../../backend/parser/gram.y
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
timestamp.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/timestamp.c -MMD
-MP -MF .deps/timestamp.Po
/usr/bin/flex -o'pgc.c'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/pgc.l
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o type.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/type.c -MMD -MP -MF
.deps/type.Po
rm -f pgstrcasecmp.c && ln -s
/home/andrew/pgl/pg_head/src/port/pgstrcasecmp.c .
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
interval.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/interval.c -MMD
-MP -MF .deps/interval.Po
( echo '{ global:'; gawk '/^[^#]/ {printf "%s;\n",$1}'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/exports.txt;
echo ' local: *; };' ) >exports.list
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
pgstrcasecmp.o pgstrcasecmp.c -MMD -MP -MF .deps/pgstrcasecmp.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o ecpg.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/ecpg.c -MMD -MP -MF
.deps/ecpg.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o output.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/output.c -MMD -MP
-MF .deps/output.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o parser.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parser.c -MMD -MP
-MF .deps/parser.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o keywords.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/keywords.c -MMD -MP
-MF .deps/keywords.Po
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parser.c:25:21:
fatal error: preproc.h: No such file or directory
compilation terminated.
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/keywords.c:20:21:
fatal error: preproc.h: No such file or directory
compilation terminated.
make[4]: *** [parser.o] Error 1
make[4]: *** Waiting for unfinished jobs....
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -shared -Wl,-soname,libpgtypes.so.3
-Wl,--version-script=exports.list -o libpgtypes.so.3.4 numeric.o
datetime.o common.o dt_common.o timestamp.o interval.o pgstrcasecmp.o
-L../../../../src/port -Wl,--as-needed
-Wl,-rpath,'/home/andrew/pgl/inst.testpar.5701/lib',--enable-new-dtags -lm
ar crs libpgtypes.a numeric.o datetime.o common.o dt_common.o
timestamp.o interval.o pgstrcasecmp.o
ranlib libpgtypes.a
rm -f libpgtypes.so.3
ln -s libpgtypes.so.3.4 libpgtypes.so.3
rm -f libpgtypes.so
ln -s libpgtypes.so.3.4 libpgtypes.so
make[4]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
make -C ecpglib all
make[4]: *** [keywords.o] Error 1
make[4]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/preproc'
make[3]: *** [all-preproc-recurse] Error 2
make[3]: *** Waiting for unfinished jobs....
make[4]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/ecpglib'
make -C ../../../../src/interfaces/libpq all
make -C ../../../../src/interfaces/ecpg/pgtypeslib all
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
execute.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/execute.c
-MMD -MP -MF .deps/execute.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
typename.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/typename.c -MMD -MP
-MF .deps/typename.Po
make[5]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
make[5]: Nothing to be done for `all'.
make[5]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
descriptor.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/descriptor.c -MMD
-MP -MF .deps/descriptor.Po
make[5]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/libpq'
make[5]: Nothing to be done for `all'.
make[5]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/libpq'
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
sqlda.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/sqlda.c
-MMD -MP -MF .deps/sqlda.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o data.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/data.c -MMD -MP -MF
.deps/data.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
error.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/error.c
-MMD -MP -MF .deps/error.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
prepare.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/prepare.c
-MMD -MP -MF .deps/prepare.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
memory.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/memory.c
-MMD -MP -MF .deps/memory.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
connect.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/connect.c
-MMD -MP -MF .deps/connect.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o misc.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/misc.c -MMD -MP -MF
.deps/misc.Po
rm -f path.c && ln -s /home/andrew/pgl/pg_head/src/port/path.c .
rm -f pgstrcasecmp.c && ln -s
/home/andrew/pgl/pg_head/src/port/pgstrcasecmp.c .
rm -f strlcpy.c && ln -s /home/andrew/pgl/pg_head/src/port/strlcpy.c .
rm -f thread.c && ln -s /home/andrew/pgl/pg_head/src/port/thread.c .
( echo '{ global:'; gawk '/^[^#]/ {printf "%s;\n",$1}'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/exports.txt; echo '
local: *; };' ) >exports.list
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o path.o
path.c -MMD -MP -MF .deps/path.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
pgstrcasecmp.o pgstrcasecmp.c -MMD -MP -MF .deps/pgstrcasecmp.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
strlcpy.o strlcpy.c -MMD -MP -MF .deps/strlcpy.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
thread.o thread.c -MMD -MP -MF .deps/thread.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -shared -Wl,-soname,libecpg.so.6
-Wl,--version-script=exports.list -o libecpg.so.6.5 execute.o typename.o
descriptor.o sqlda.o data.o error.o prepare.o memory.o connect.o misc.o
path.o pgstrcasecmp.o strlcpy.o thread.o -L../../../../src/port
-Wl,--as-needed
-Wl,-rpath,'/home/andrew/pgl/inst.testpar.5701/lib',--enable-new-dtags
-L../pgtypeslib -lpgtypes -L../../../../src/interfaces/libpq -lpq -lm
-lpthread
ar crs libecpg.a execute.o typename.o descriptor.o sqlda.o data.o
error.o prepare.o memory.o connect.o misc.o path.o pgstrcasecmp.o
strlcpy.o thread.o
ranlib libecpg.a
rm -f libecpg.so.6
ln -s libecpg.so.6.5 libecpg.so.6
rm -f libecpg.so
ln -s libecpg.so.6.5 libecpg.so
make[4]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/ecpglib'
make[3]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg'
make[2]: *** [all-ecpg-recurse] Error 2
make[2]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/interfaces'
make[1]: *** [all-interfaces-recurse] Error 2
make[1]: Leaving directory `/home/pgl/npgl/vpath.testpar/src'
make: *** [all-src-recurse] Error 2

Andrew Dunstan <andrew@dunslane.net> writes:

Well, it looks like it's always failing on ecpg, with preproc.h not
being made in the right order. Here is the last bit of a make log
starting from when it starts on ecpg. This is pretty repeatable.

Hmph. I can't reproduce it at all on my Fedora 16 box. What version
of make are you using?

regards, tom lane

On 09/07/2012 08:43 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Well, it looks like it's always failing on ecpg, with preproc.h not
being made in the right order. Here is the last bit of a make log
starting from when it starts on ecpg. This is pretty repeatable.

Hmph. I can't reproduce it at all on my Fedora 16 box. What version
of make are you using?

$ make -v
GNU Make 3.82
Built for x86_64-redhat-linux-gnu

cheers

andrew

On 09/07/2012 09:55 PM, Andrew Dunstan wrote:

On 09/07/2012 08:43 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Well, it looks like it's always failing on ecpg, with preproc.h not
being made in the right order. Here is the last bit of a make log
starting from when it starts on ecpg. This is pretty repeatable.

Hmph. I can't reproduce it at all on my Fedora 16 box. What version
of make are you using?

$ make -v
GNU Make 3.82
Built for x86_64-redhat-linux-gnu

OK, I just tried on a different F16 machine and it didn't happen. I
wonder what's different.

cheers

andrew

On 09/07/2012 10:46 PM, Andrew Dunstan wrote:

On 09/07/2012 09:55 PM, Andrew Dunstan wrote:

On 09/07/2012 08:43 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Well, it looks like it's always failing on ecpg, with preproc.h not
being made in the right order. Here is the last bit of a make log
starting from when it starts on ecpg. This is pretty repeatable.

Hmph. I can't reproduce it at all on my Fedora 16 box. What version
of make are you using?

$ make -v
GNU Make 3.82
Built for x86_64-redhat-linux-gnu

OK, I just tried on a different F16 machine and it didn't happen. I
wonder what's different.

This seems totally stupid, but it happens when the path to the current
directory includes a cross-device symlink. If I cd following the link,
then this effect doesn't happen. Weird.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

This seems totally stupid, but it happens when the path to the current
directory includes a cross-device symlink. If I cd following the link,
then this effect doesn't happen. Weird.

Huh. So maybe a gmake bug, or maybe there's something wrong with our
make rules for the case that `pwd` doesn't match what gmake thinks the
current path is. Could you specify the setup more fully?

regards, tom lane

On 09/08/2012 11:06 AM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

This seems totally stupid, but it happens when the path to the current
directory includes a cross-device symlink. If I cd following the link,
then this effect doesn't happen. Weird.

Huh. So maybe a gmake bug, or maybe there's something wrong with our
make rules for the case that `pwd` doesn't match what gmake thinks the
current path is. Could you specify the setup more fully?

Scratch that theory, that was just a transient. If anything it looks
like it is related to system load. When almost nothing was running on
the machine it worked fine. When I started up a Browser and an MUA the
problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
average around 0.4 right now. I'm a bit perplexed.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

Scratch that theory, that was just a transient. If anything it looks
like it is related to system load. When almost nothing was running on
the machine it worked fine. When I started up a Browser and an MUA the
problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
average around 0.4 right now. I'm a bit perplexed.

Hm ... you weren't using the -l (--max-load) option were you? That
would make system load affect gmake's scheduling. Although it's clear
when I test it that it is waiting for the bison run to finish before
launching the dependent builds, so it still seems like it must be a bug
if your copy isn't waiting for that.

regards, tom lane

On 09/08/2012 04:46 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Scratch that theory, that was just a transient. If anything it looks
like it is related to system load. When almost nothing was running on
the machine it worked fine. When I started up a Browser and an MUA the
problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
average around 0.4 right now. I'm a bit perplexed.

Hm ... you weren't using the -l (--max-load) option were you? That
would make system load affect gmake's scheduling. Although it's clear
when I test it that it is waiting for the bison run to finish before
launching the dependent builds, so it still seems like it must be a bug
if your copy isn't waiting for that.

No. just "make -j 4"

And it's the stock Fedora build of make.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

And it's the stock Fedora build of make.

Which Fedora branch exactly?

The package version I was trying to reproduce with here is
make-3.82-8.fc16.x86_64.

regards, tom lane

On 09/08/2012 04:54 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

And it's the stock Fedora build of make.

Which Fedora branch exactly?

The package version I was trying to reproduce with here is
make-3.82-8.fc16.x86_64.

Same:

$ rpm -q make
make-3.82-8.fc16.x86_64

cheers

andrew

On 09/08/2012 04:52 PM, Andrew Dunstan wrote:

On 09/08/2012 04:46 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Scratch that theory, that was just a transient. If anything it looks
like it is related to system load. When almost nothing was running on
the machine it worked fine. When I started up a Browser and an MUA the
problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
average around 0.4 right now. I'm a bit perplexed.

Hm ... you weren't using the -l (--max-load) option were you? That
would make system load affect gmake's scheduling. Although it's clear
when I test it that it is waiting for the bison run to finish before
launching the dependent builds, so it still seems like it must be a bug
if your copy isn't waiting for that.

No. just "make -j 4"

And it's the stock Fedora build of make.

I have just repeated this on an absolutely fresh up to date F17 machine,
with no symlink stuff in play.

Steps to recreate:

CC="ccache gcc" ../postgres/configure --enable-depend --enable-debug
--enable-cassert --with-perl --with-python --with-tcl --with-libxml
--with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap
make -j 4

cheers

andrew

q