Add documentation for coverage reports with meson

Started by Michael Paquieralmost 3 years ago5 messages

michael@paquier.xyz

almost 3 years ago

1 attachment(s)

Hi all,

I have mentioned on a different thread of -docs that we have no
documentation to achieve $subject, so attached is a patch to add
something. This can be done with the following steps:
meson setup -Db_coverage=true .. blah
ninja
meson test
ninja coverage-html

As far as I can see, there is no option to generate anything else than
a HTML report? This portion is telling the contrary, still it does
not seem to work here and ninja does the job with coverage-html or
coverage as only available targets:
https://mesonbuild.com/howtox.html#producing-a-coverage-report

Side issue: the current code generates no reports for the files that
are automatically generated in src/backend/nodes/, which are actually
part of src/include/ for a meson build. I have not looked into that
yet.

Thoughts?
--
Michael

Attachments:

meson-coverage-docs.patchtext/x-diff; charset=us-asciiDownload

diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index a08c7a78af..dd9a004a6d 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -825,53 +825,76 @@ PG_TEST_NOCLEAN=1 make -C src/bin/pg_dump check
     instrumentation, so that it becomes possible to examine which
     parts of the code are covered by the regression tests or any other
     test suite that is run with the code.  This is currently supported
-    when compiling with GCC, and it requires the <command>gcov</command>
-    and <command>lcov</command> programs.
+    when compiling with GCC, and it requires the <command>gcov</command>,
+    <command>lcov</command> and <command>genhtml</command> programs.
    </para>
 
-   <para>
-    A typical workflow looks like this:
+   <sect2 id="regress-coverage-configure">
+    <title>Coverage with <filename>configure</filename></title>
+    <para>
+     A typical workflow looks like this:
 <screen>
 ./configure --enable-coverage ... OTHER OPTIONS ...
 make
 make check # or other test suite
 make coverage-html
 </screen>
-    Then point your HTML browser
-    to <filename>coverage/index.html</filename>.
-   </para>
+     Then point your HTML browser
+     to <filename>coverage/index.html</filename>.
+    </para>
 
-   <para>
-    If you don't have <command>lcov</command> or prefer text output over an
-    HTML report, you can run
+    <para>
+     If you don't have <command>lcov</command> or prefer text output over an
+     HTML report, you can run
 <screen>
 make coverage
 </screen>
-    instead of <literal>make coverage-html</literal>, which will
-    produce <filename>.gcov</filename> output files for each source file
-    relevant to the test.  (<literal>make coverage</literal> and <literal>make
-    coverage-html</literal> will overwrite each other's files, so mixing them
-    might be confusing.)
-   </para>
+     instead of <literal>make coverage-html</literal>, which will
+     produce <filename>.gcov</filename> output files for each source file
+     relevant to the test.  (<literal>make coverage</literal> and <literal>make
+     coverage-html</literal> will overwrite each other's files, so mixing them
+     might be confusing.)
+    </para>
 
-   <para>
-    You can run several different tests before making the coverage report;
-    the execution counts will accumulate.  If you want
-    to reset the execution counts between test runs, run:
+    <para>
+     You can run several different tests before making the coverage report;
+     the execution counts will accumulate.  If you want
+     to reset the execution counts between test runs, run:
 <screen>
 make coverage-clean
 </screen>
-   </para>
+    </para>
 
-   <para>
-    You can run the <literal>make coverage-html</literal> or <literal>make
-    coverage</literal> command in a subdirectory if you want a coverage
-    report for only a portion of the code tree.
-   </para>
+    <para>
+     You can run the <literal>make coverage-html</literal> or <literal>make
+     coverage</literal> command in a subdirectory if you want a coverage
+     report for only a portion of the code tree.
+    </para>
 
-   <para>
-    Use <literal>make distclean</literal> to clean up when done.
-   </para>
+    <para>
+     Use <literal>make distclean</literal> to clean up when done.
+    </para>
+   </sect2>
+
+   <sect2 id="regress-coverage-meson">
+    <title>Coverage with <filename>meson</filename></title>
+    <para>
+     A typical workflow looks like this:
+<screen>
+meson setup -Db_coverage=true ... OTHER OPTIONS ...
+ninja
+meson test
+ninja coverage-html
+</screen>
+     Then point your HTML browser
+     to <filename>./meson-logs/coveragereport/index.html</filename>.
+    </para>
+
+    <para>
+     You can run several different tests before making the coverage report;
+     the execution counts will accumulate.
+    </para>
+   </sect2>
   </sect1>
 
 </chapter>

Peter Eisentraut

peter.eisentraut@enterprisedb.com

almost 3 years ago

In reply to: Michael Paquier (#1)

Re: Add documentation for coverage reports with meson

On 28.02.23 09:49, Michael Paquier wrote:

-    when compiling with GCC, and it requires the <command>gcov</command>
-    and <command>lcov</command> programs.
+    when compiling with GCC, and it requires the <command>gcov</command>,
+    <command>lcov</command> and <command>genhtml</command> programs.

genhtml is part of the lcov package. I think it would be confusing to
mention it explicitly, since you won't be able to find it as something
to install. Maybe leave the original list and change "programs" to
"packages"?

-   <para>
-    A typical workflow looks like this:
+   <sect2 id="regress-coverage-configure">
+    <title>Coverage with <filename>configure</filename></title>
+    <para>
+     A typical workflow looks like this:

In the installation chapter we use titles like "Building and
Installation with Autoconf and Make" and "Building and Installation with
Meson". We should use analogous wordings here.

+    <para>
+     A typical workflow looks like this:
+<screen>
+meson setup -Db_coverage=true ... OTHER OPTIONS ...
+ninja
+meson test
+ninja coverage-html
+</screen>
+     Then point your HTML browser
+     to <filename>./meson-logs/coveragereport/index.html</filename>.
+    </para>

This ignores which directory you have to be in. The meson calls have to
be at the top level, the ninja calls have to be in the build directory.
We should be more precise here, otherwise someone trying this will find
that it doesn't work.

Personally I use "meson compile" instead of "ninja"; I'm not sure what
the best recommendation is, but that least that way all the initial
commands are "meson something" instead of going back and forth.

Michael Paquier

michael@paquier.xyz

almost 3 years ago

In reply to: Peter Eisentraut (#2)

1 attachment(s)

Re: Add documentation for coverage reports with meson

On Fri, Mar 03, 2023 at 10:10:15AM +0100, Peter Eisentraut wrote:

genhtml is part of the lcov package. I think it would be confusing to
mention it explicitly, since you won't be able to find it as something to
install. Maybe leave the original list and change "programs" to "packages"?

Makes sense.

In the installation chapter we use titles like "Building and Installation
with Autoconf and Make" and "Building and Installation with Meson". We
should use analogous wordings here.

OK, changed to something like that.

This ignores which directory you have to be in. The meson calls have to be
at the top level, the ninja calls have to be in the build directory. We
should be more precise here, otherwise someone trying this will find that it
doesn't work.

Hmm. I can see that it is possible to pass the repository to move to
with -C, still it is simpler to move into the build repository.

Personally I use "meson compile" instead of "ninja"; I'm not sure what the
best recommendation is, but that least that way all the initial commands are
"meson something" instead of going back and forth.

Using meson compile is fine by me for the docs. Note that I cannot
see an option with meson to do coverage reports, and my environment
uses 1.0.1. Only ninja handles that.

Updated version attached.
--
Michael

Attachments:

meson-coverage-docs-v2.patchtext/x-diff; charset=us-asciiDownload

diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index a08c7a78af..344c834280 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -825,53 +825,77 @@ PG_TEST_NOCLEAN=1 make -C src/bin/pg_dump check
     instrumentation, so that it becomes possible to examine which
     parts of the code are covered by the regression tests or any other
     test suite that is run with the code.  This is currently supported
-    when compiling with GCC, and it requires the <command>gcov</command>
-    and <command>lcov</command> programs.
+    when compiling with GCC, and it requires the <literal>gcov</literal>
+    and <literal>lcov</literal> packages.
    </para>
 
-   <para>
-    A typical workflow looks like this:
+   <sect2 id="regress-coverage-configure">
+    <title>Coverage with Autoconf and Make</title>
+    <para>
+     A typical workflow looks like this:
 <screen>
 ./configure --enable-coverage ... OTHER OPTIONS ...
 make
 make check # or other test suite
 make coverage-html
 </screen>
-    Then point your HTML browser
-    to <filename>coverage/index.html</filename>.
-   </para>
+     Then point your HTML browser
+     to <filename>coverage/index.html</filename>.
+    </para>
 
-   <para>
-    If you don't have <command>lcov</command> or prefer text output over an
-    HTML report, you can run
+    <para>
+     If you don't have <command>lcov</command> or prefer text output over an
+     HTML report, you can run
 <screen>
 make coverage
 </screen>
-    instead of <literal>make coverage-html</literal>, which will
-    produce <filename>.gcov</filename> output files for each source file
-    relevant to the test.  (<literal>make coverage</literal> and <literal>make
-    coverage-html</literal> will overwrite each other's files, so mixing them
-    might be confusing.)
-   </para>
+     instead of <literal>make coverage-html</literal>, which will
+     produce <filename>.gcov</filename> output files for each source file
+     relevant to the test.  (<literal>make coverage</literal> and <literal>make
+     coverage-html</literal> will overwrite each other's files, so mixing them
+     might be confusing.)
+    </para>
 
-   <para>
-    You can run several different tests before making the coverage report;
-    the execution counts will accumulate.  If you want
-    to reset the execution counts between test runs, run:
+    <para>
+     You can run several different tests before making the coverage report;
+     the execution counts will accumulate.  If you want
+     to reset the execution counts between test runs, run:
 <screen>
 make coverage-clean
 </screen>
-   </para>
+    </para>
 
-   <para>
-    You can run the <literal>make coverage-html</literal> or <literal>make
-    coverage</literal> command in a subdirectory if you want a coverage
-    report for only a portion of the code tree.
-   </para>
+    <para>
+     You can run the <literal>make coverage-html</literal> or <literal>make
+     coverage</literal> command in a subdirectory if you want a coverage
+     report for only a portion of the code tree.
+    </para>
 
-   <para>
-    Use <literal>make distclean</literal> to clean up when done.
-   </para>
+    <para>
+     Use <literal>make distclean</literal> to clean up when done.
+    </para>
+   </sect2>
+
+   <sect2 id="regress-coverage-meson">
+    <title>Coverage with Meson</title>
+    <para>
+     A typical workflow looks like this:
+<screen>
+meson setup -Db_coverage=true ... OTHER OPTIONS ... builddir/
+cd builddir/
+meson compile
+meson test
+ninja coverage-html
+</screen>
+     Then point your HTML browser
+     to <filename>./meson-logs/coveragereport/index.html</filename>.
+    </para>
+
+    <para>
+     You can run several different tests before making the coverage report;
+     the execution counts will accumulate.
+    </para>
+   </sect2>
   </sect1>
 
 </chapter>

Peter Eisentraut

peter.eisentraut@enterprisedb.com

almost 3 years ago

In reply to: Michael Paquier (#3)

Re: Add documentation for coverage reports with meson

On 03.03.23 12:12, Michael Paquier wrote:

+<screen>
+meson setup -Db_coverage=true ... OTHER OPTIONS ... builddir/
+cd builddir/
+meson compile
+meson test
+ninja coverage-html
+</screen>

The "cd" command needs to be moved after the meson commands, and the
meson commands need to have a -C builddir option. So it should be like

<screen>
meson setup -Db_coverage=true ... OTHER OPTIONS ... builddir/
meson compile -C builddir
meson test -C builddir
cd builddir/
ninja coverage-html
</screen>

Otherwise, this looks good to me.

Michael Paquier

michael@paquier.xyz

almost 3 years ago

In reply to: Peter Eisentraut (#4)

Re: Add documentation for coverage reports with meson

On Wed, Mar 08, 2023 at 05:23:48PM +0100, Peter Eisentraut wrote:

The "cd" command needs to be moved after the meson commands, and the meson
commands need to have a -C builddir option.

Still that's not mandatory, is it? The compile and test commands of
meson work as well if you are located at the root of the build
directory, AFAIK.

So it should be like

<screen>
meson setup -Db_coverage=true ... OTHER OPTIONS ... builddir/
meson compile -C builddir
meson test -C builddir
cd builddir/
ninja coverage-html
</screen>

Otherwise, this looks good to me.

Anyway, this works as well and I don't have any arguments against
that. So I have used your flow, and applied the patch. I have
actually switched my own scripts to rely more on -C, removing direct
calls to ninja ;p

Thanks for the feedback.
--
Michael