Question about make coverage-html

On 27/10/2020 10:09, Peter Smith wrote:

Hi hackers.

The example of test coverage in the documentation [1] works as advertised.

But I wanted to generate test coverage results only of some TAP tests
in src/test/subscription.

The documentation [1] also says "The make commands also work in
subdirectories." so I tried running them all in that folder.

However, when I run "make coverage-html" in that subdirectory
src/test/subscription it does not work:

=====
[postgres@CentOS7-x64 subscription]$ make coverage-html
/usr/local/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i
-d . -d . -o lcov_base.info
geninfo: WARNING: no .gcno files found in . - skipping!
geninfo: WARNING: no .gcno files found in . - skipping!
/usr/local/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -d .
-d . -o lcov_test.info
geninfo: WARNING: no .gcda files found in . - skipping!
geninfo: WARNING: no .gcda files found in . - skipping!
rm -rf coverage
/usr/local/bin/genhtml -q --legend -o coverage --title='PostgreSQL
14devel' --num-spaces=4 --prefix='/home/postgres/oss_postgres_2PC'
lcov_base.info lcov_test.info
genhtml: ERROR: no valid records found in tracefile lcov_base.info
make: *** [coverage-html-stamp] Error 255
[postgres@CentOS7-x64 subscription]$
=====

OTOH, running the "make coverage-html" at the top folder after running
my TAP tests does give the desired coverage results.

~

QUESTION:

Was that documentation [1] just being misleading by saying it can work
in the subdirectories?
e.g. Are you only supposed to run "make coverage-html" from the top folder?

Or is it supposed to work but I did something wrong?

Running "make coverage-html" in src/test/subscription doesn't work,
because there is no C code in that directory.

Creating a coverage report is a two-step process. First, you run the
test you're interested in, with "make check" or similar. Then you create
a report for the source files you're interested in, with "make
coverage-html". You can run these commands in different subdirectories.

In this case, you want to do "cd src/test/subscription; make check", to
run those TAP tests, and then run "make coverage-html" from the top
folder. Or if you wanted to create coverage report that covers only
replication-related source code, for example, you could run it in the
src/backend/replication directory ("cd src/backend/replication; make
coverage-html").

- Heikki

Heikki Linnakangas <hlinnaka@iki.fi> writes:

On 27/10/2020 10:09, Peter Smith wrote:

The documentation [1] also says "The make commands also work in
subdirectories." so I tried running them all in that folder.
However, when I run "make coverage-html" in that subdirectory
src/test/subscription it does not work:

Creating a coverage report is a two-step process. First, you run the
test you're interested in, with "make check" or similar. Then you create
a report for the source files you're interested in, with "make
coverage-html". You can run these commands in different subdirectories.

In this case, you want to do "cd src/test/subscription; make check", to
run those TAP tests, and then run "make coverage-html" from the top
folder. Or if you wanted to create coverage report that covers only
replication-related source code, for example, you could run it in the
src/backend/replication directory ("cd src/backend/replication; make
coverage-html").

I agree with the OP that the documentation is a bit vague here.
I think (maybe I'm wrong) that it's clear enough that you can run
whichever test case(s) you want, but this behavior of generating a
partial coverage report is less clear. Maybe instead of

The "make" commands also work in subdirectories.

we could say

You can run the "make coverage-html" command in a subdirectory
if you want a coverage report for only a portion of the code tree.

regards, tom lane

Creating a coverage report is a two-step process. First, you run the
test you're interested in, with "make check" or similar. Then you create
a report for the source files you're interested in, with "make
coverage-html". You can run these commands in different subdirectories.

In this case, you want to do "cd src/test/subscription; make check", to
run those TAP tests, and then run "make coverage-html" from the top
folder. Or if you wanted to create coverage report that covers only
replication-related source code, for example, you could run it in the
src/backend/replication directory ("cd src/backend/replication; make
coverage-html").

I agree with the OP that the documentation is a bit vague here.
I think (maybe I'm wrong) that it's clear enough that you can run
whichever test case(s) you want, but this behavior of generating a
partial coverage report is less clear. Maybe instead of

The "make" commands also work in subdirectories.

we could say

You can run the "make coverage-html" command in a subdirectory
if you want a coverage report for only a portion of the code tree.

Thank you for the clarifications and the updated documentation.

Kind Regards,
Peter Smith
Fujitsu Australia