make coverage-html on OS X

On 10/13/16 4:03 PM, Jim Nasby wrote:

I've slowly stripped away my normal build environment; I'm now using
baseline gcc (no ccache) with the following ./configure... anyone have
any ideas how to debug/fix this?

Which gcc are you using?

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/13/16 11:13 PM, Peter Eisentraut wrote:

On 10/13/16 4:03 PM, Jim Nasby wrote:

I've slowly stripped away my normal build environment; I'm now using
baseline gcc (no ccache) with the following ./configure... anyone have
any ideas how to debug/fix this?

Which gcc are you using?

gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr
--with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.0.0 (clang-800.0.38)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

FWIW, I normally do

CC='ccache clang -Qunused-arguments -fcolor-diagnostics'

which was also failing; I've now unset CC (and verified plain gcc is in
use).
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/14/16 11:50 AM, Jim Nasby wrote:

On 10/13/16 11:13 PM, Peter Eisentraut wrote:

On 10/13/16 4:03 PM, Jim Nasby wrote:

I've slowly stripped away my normal build environment; I'm now using
baseline gcc (no ccache) with the following ./configure... anyone have
any ideas how to debug/fix this?

Which gcc are you using?

gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr
--with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.0.0 (clang-800.0.38)

I would try it with something that is not the system compiler. For
example, a gcc build from Homebrew.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/17/16 7:22 AM, Peter Eisentraut wrote:

On 10/14/16 11:50 AM, Jim Nasby wrote:

On 10/13/16 11:13 PM, Peter Eisentraut wrote:

On 10/13/16 4:03 PM, Jim Nasby wrote:

I've slowly stripped away my normal build environment; I'm now using
baseline gcc (no ccache) with the following ./configure... anyone have
any ideas how to debug/fix this?

Which gcc are you using?

gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr
--with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.0.0 (clang-800.0.38)

I would try it with something that is not the system compiler. For
example, a gcc build from Homebrew.

Well, that got me closer, but it's still blowing up on libpq:

genhtml: ERROR: no valid records found in tracefile
./src/interfaces/libpq/lcov.info

That file has 21 lines of nothing but "TN:". There's also a few
complaints about not being able to find object file for ___gcov_execl
and ___gcov_fork. Note the 3 _s, not 2.

I've attached a build log if anyone wants to take a look...

decibel@decina:[12:08]~/pgsql/HEAD (master $%=)$gcov --version
gcov (MacPorts gcc49 4.9.4_0) 4.9.4
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.

decibel@decina:[12:14]~/pgsql/HEAD (master $%=)$gcc --version
gcc (MacPorts gcc49 4.9.4_0) 4.9.4
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461

On 10/27/16 12:27 PM, Jim Nasby wrote:

I've attached a build log if anyone wants to take a look...

BTW, while the log shows Apple's make in use I also tried with gmake out
of macports with the same results.
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/27/16 1:27 PM, Jim Nasby wrote:

Well, that got me closer, but it's still blowing up on libpq:

genhtml: ERROR: no valid records found in tracefile
./src/interfaces/libpq/lcov.info

I have seen similar problems when I use a gcov that does not match the gcc.

I was able to run it successfully using CC=gcc-6 and GCOV=gcov-6 from
Homebrew.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

tl;dr: It's critical that you actually do a make install, or at least it
is if you've set --prefix with configure. If you don't, then even if you
do make check you'le going to get the *installed* libpq, and not the
*built* libpq.

Also, looks like there's a race between the .info and .c.gcov targets
with parallel make; I'm wondering if there's a way to fix that by not
allowing parallelism in each directory...? (Presumably the issue is the
rm *.gcov). It'd be nice to fix this because -j speeds up coverage-html
a lot, even with just 2 cores.

On 10/27/16 9:38 PM, Peter Eisentraut wrote:

On 10/27/16 1:27 PM, Jim Nasby wrote:

Well, that got me closer, but it's still blowing up on libpq:

genhtml: ERROR: no valid records found in tracefile
./src/interfaces/libpq/lcov.info

I have seen similar problems when I use a gcov that does not match the gcc.

I switched back to macports gcc6, verified version correctness, and got
further (though still broken). Looking through the build log:

Processing fe-lobj.gcda
/Users/decibel/pgsql/HEAD/src/interfaces/libpq/fe-lobj.gcda:stamp
mismatch with notes file
geninfo: WARNING: gcov did not create any files for
/Users/decibel/pgsql/HEAD/src/interfaces/libpq/fe-lobj.gcda!
...
gcov -b -f -p -o . fe-lobj.c >fe-lobj.c.gcov.out
./fe-lobj.gcda:stamp mismatch with notes file

First hit[1] on "google:gcov mismatch with notes file" led me to this
hexdump command (picking a libpq file at random...)

hexdump -e '"%x\n"' -s8 -n4 fe-lobj.gcda
6929786
hexdump -e '"%x\n"' -s8 -n4 fe-lobj.gcno
d93a160

Second hit[2] gives a better explaination: the files were rebuilt a
second time. While I don't see that happening in the log, his example is
from building a shared library, so I'm wondering if that's got something
to do with this.

I went into src/interfaces/libpq, did rm *.gc* lcov.info. After doing
that, a second run of make coverage-html worked fine.

I'm wondering if there's some magic involved in coverage with shared
libraries...

Actually, after a bunch of other experiments I ran those hexdump
commands again; the value for .gcno has changed, but the value for .gcda
hasn't. So I'm wondering if the base system libpq is getting pulled in.
Sure enough, if I also do make install the timestamp matches. Presumably
this is all due to having set --prefix with configure.

I was able to run it successfully using CC=gcc-6 and GCOV=gcov-6 from
Homebrew.

I tried that as well; it didn't work either. I didn't run a diff, but it
appeared to be doing the same thing that macports gcc6 was.

1:
http://stackoverflow.com/questions/27276155/gcov-generates-empty-coverage-for-c
2: http://stackoverflow.com/a/26061575/4196282
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/29/16 3:31 PM, Jim Nasby wrote:

tl;dr: It's critical that you actually do a make install, or at least it
is if you've set --prefix with configure. If you don't, then even if you
do make check you'le going to get the *installed* libpq, and not the
*built* libpq.

Actually, sometimes that isn't even enough. I had one checkout that was
refusing to successfully build coverage (producing an old .gcda for
*everything*) until I rm'd the entire install directory (make uninstall
was not enough). I also killed the running postmaster as part of that,
so it's possible that's what was causing the problems.
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/29/16 4:31 PM, Jim Nasby wrote:

tl;dr: It's critical that you actually do a make install, or at least it
is if you've set --prefix with configure. If you don't, then even if you
do make check you'le going to get the *installed* libpq, and not the
*built* libpq.

I was not able to reproduce this. I suspect that this would happen if
you don't disable System Integrity Protection, because if SIP is on,
then the environment variable DYLD_LIBRARY_PATH is disabled, and a
regression test run would be prone to use a mix of source tree and
install tree components. Often, this is not a problem, but if your
source tree is built with different options than the previous install,
then it could make a mess. Building with coverage instrumentation is
obviously one case where the build options are quite different.

Also, looks like there's a race between the .info and .c.gcov targets
with parallel make; I'm wondering if there's a way to fix that by not
allowing parallelism in each directory...? (Presumably the issue is the
rm *.gcov). It'd be nice to fix this because -j speeds up coverage-html
a lot, even with just 2 cores.

I was not able to reproduce this. The make rules look reasonable to me
for parallelism.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers