Обсуждение: Having problems generating a code coverage report
Hi hackers,
Recently I tried to build a code coverage report according to the
documentation [1]. When using the following command:
```
time sh -c 'git clean -dfx && meson setup --buildtype debug
-DPG_TEST_EXTRA="kerberos ldap ssl" -Db_coverage=true -Dldap=disabled
-Dssl=openssl -Dcassert=true -Dtap_tests=enabled
-Dprefix=/home/eax/pginstall build && ninja -C build &&
PG_TEST_EXTRA=1 meson test -C build && ninja -C build coverage-html'
```
... I get:
```
geninfo: ERROR: Unexpected negative count '-3' for
/home/eax/projects/c/postgresql/src/port/snprintf.c:532.
Perhaps you need to compile with '-fprofile-update=atomic
(use "geninfo --ignore-errors negative ..." to bypass this error)
Traceback (most recent call last):
File "/usr/bin/meson", line 40, in <module>
sys.exit(mesonmain.main())
^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
294, in main
return run(sys.argv[1:], launcher)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
282, in run
return run_script_command(args[1], args[2:])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
223, in run_script_command
return module.run(script_args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 206, in run
return coverage(options.outputs, options.source_root,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 123, in coverage
subprocess.check_call([lcov_exe,
File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['lcov', '--directory',
'/home/eax/projects/c/postgresql/build', '--capture', '--output-file',
'/home/eax/projects/c/postgresql/build/meson-logs/coverage.info.run',
'--no-checksum', '--rc', 'branch_coverage=1']' returned non-zero exit
status 1.
ninja: build stopped: subcommand failed.
```
If I add -fprofile-update=atomic as suggested:
```
time CFLAGS="-fprofile-update=atomic"
CXXFLAGS="-fprofile-update=atomic" sh -c 'git clean -dfx && meson
setup --buildtype debug -DPG_TEST_EXTRA="kerberos ldap ssl"
-Db_coverage=true -Dldap=disabled -Dssl=openssl -Dcassert=true
-Dtap_tests=enabled -Dprefix=/home/eax/pginstall build && ninja -C
build && PG_TEST_EXTRA=1 meson test -C build && ninja -C build
coverage-html'
```
... I get:
```
genhtml: ERROR: duplicate merge record src/include/catalog
Traceback (most recent call last):
File "/usr/bin/meson", line 40, in <module>
sys.exit(mesonmain.main())
^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
294, in main
return run(sys.argv[1:], launcher)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
282, in run
return run_script_command(args[1], args[2:])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
223, in run_script_command
return module.run(script_args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 206, in run
return coverage(options.outputs, options.source_root,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 150, in coverage
subprocess.check_call([genhtml_exe,
File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['genhtml', '--prefix',
'/home/eax/projects/c/postgresql/build', '--prefix',
'/home/eax/projects/c/postgresql', '--output-directory',
'/home/eax/projects/c/postgresql/build/meson-logs/coveragereport',
'--title', 'Code coverage', '--legend', '--show-details',
'--branch-coverage',
'/home/eax/projects/c/postgresql/build/meson-logs/coverage.info']'
returned non-zero exit status 1.
ninja: build stopped: subcommand failed.
```
I'm using Xubuntu 24.04 LTS and my lcov version is:
```
$ lcov --version
lcov: LCOV version 2.0-1
```
I tried using Autotools with the same results. Pretty confident it
worked before. I'm wondering if anyone else experienced this lately
and/or knows a workaround.
[1]: https://www.postgresql.org/docs/current/regress-coverage.html
--
Best regards,
Aleksander Alekseev
On Wed, Oct 30, 2024 at 5:21 PM Aleksander Alekseev <aleksander@timescale.com> wrote: > I'm using Xubuntu 24.04 LTS and my lcov version is: > > ``` > $ lcov --version > lcov: LCOV version 2.0-1 > ``` I use Debian unstable for most of my day to day work. Apparently Debian unstable has exactly the same version of lcov as Ubuntu 24.04. I've also been unable to generate coverage reports for some time (at least on Debian, with LCOV version 2.0-1). -- Peter Geoghegan
Hi, > On Wed, Oct 30, 2024 at 05:26:49PM -0400, Peter Geoghegan wrote: > > I use Debian unstable for most of my day to day work. Apparently > > Debian unstable has exactly the same version of lcov as Ubuntu 24.04. > > > > I've also been unable to generate coverage reports for some time (at > > least on Debian, with LCOV version 2.0-1). > > So do I, for both the debian SID and the lcov parts. I downgraded to lcov 1.16 [1] and it helped. This is merely a workaround of course, not a long-time solution. [1]: https://github.com/linux-test-project/lcov/releases/tag/v1.16 -- Best regards, Aleksander Alekseev
On Thu, Oct 31, 2024 at 10:02 PM Aleksander Alekseev <aleksander@timescale.com> wrote: > > Hi, > > > On Wed, Oct 30, 2024 at 05:26:49PM -0400, Peter Geoghegan wrote: > > > I use Debian unstable for most of my day to day work. Apparently > > > Debian unstable has exactly the same version of lcov as Ubuntu 24.04. > > > > > > I've also been unable to generate coverage reports for some time (at > > > least on Debian, with LCOV version 2.0-1). > > > > So do I, for both the debian SID and the lcov parts. > > I downgraded to lcov 1.16 [1] and it helped. This is merely a > workaround of course, not a long-time solution. > > [1]: https://github.com/linux-test-project/lcov/releases/tag/v1.16 > > -- my ubuntu meson version: 0.61.2, which also fails. with similar errors you've posted. from these two posts, https://github.com/mesonbuild/meson/pull/12345 https://github.com/mesonbuild/meson/issues/11995 Maybe upgrading meson can solve this problem.
Hi Jian, > > I downgraded to lcov 1.16 [1] and it helped. This is merely a > > workaround of course, not a long-time solution. > > > > [1]: https://github.com/linux-test-project/lcov/releases/tag/v1.16 > > > > -- > > my ubuntu meson version: 0.61.2, which also fails. > with similar errors you've posted. > > > from these two posts, > https://github.com/mesonbuild/meson/pull/12345 > https://github.com/mesonbuild/meson/issues/11995 > > Maybe upgrading meson can solve this problem. Thanks for the hint. I'm using Meson 1.3.2. Although it's not ancient (Feb 2024) there is 1.6.0 available. I'll try upgrading and let you know the results. -- Best regards, Aleksander Alekseev
Hi,
> Thanks for the hint. I'm using Meson 1.3.2. Although it's not ancient
> (Feb 2024) there is 1.6.0 available. I'll try upgrading and let you
> know the results.
I upgraded to Meson 1.6.0 and Lcov 2.0-1. Unfortunately it doesn't work:
```
genhtml: ERROR: no data for line:864, TLA:UNC,
file:/home/eax/projects/c/postgresql/src/bin/psql/psqlscanslash.l
(use "genhtml --ignore-errors unmapped ..." to bypass this error)
Traceback (most recent call last):
File "/home/eax/.venv/bin/meson", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/mesonmain.py",
line 293, in main
return run(sys.argv[1:], launcher)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/mesonmain.py",
line 281, in run
return run_script_command(args[1], args[2:])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/mesonmain.py",
line 222, in run_script_command
return module.run(script_args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/scripts/coverage.py",
line 208, in run
return coverage(options.outputs, options.source_root,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/scripts/coverage.py",
line 148, in coverage
subprocess.check_call([genhtml_exe,
File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['genhtml', '--prefix',
'/home/eax/projects/c/postgresql/build', '--prefix',
'/home/eax/projects/c/postgresql', '--output-directory',
'/home/eax/projects/c/postgresql/build/meson-logs/coveragereport',
'--title', 'Code coverage', '--legend', '--show-details',
'--branch-coverage',
'/home/eax/projects/c/postgresql/build/meson-logs/coverage.info']'
returned non-zero exit status 1.
ninja: build stopped: subcommand failed.
```
If I add `--ignore-errors unmapped` as suggested and execute:
```
genhtml --ignore-errors unmapped --prefix
/home/eax/projects/c/postgresql/build --prefix
/home/eax/projects/c/postgresql --output-directory
/home/eax/projects/c/postgresql/build/meson-logs/coveragereport
--title 'Code coverage' --legend --show-details --branch-coverage
/home/eax/projects/c/postgresql/build/meson-logs/coverage.info
```
... I get:
```
...
Processing file src/pl/plpython/plpy_util.c
lines=33 hit=22 functions=4 hit=4 branches=14 hit=5
Processing file src/include/catalog/index.h
lines=10 hit=10 functions=2 hit=2
genhtml: ERROR: duplicate merge record src/include/catalog
```
I didn't investigate further.
--
Best regards,
Aleksander Alekseev
Hi everyone, > I upgraded to Meson 1.6.0 and Lcov 2.0-1. Unfortunately it doesn't work: > [...] > I didn't investigate further. Lcov 2.1 and 2.2beta don't work either. -- Best regards, Aleksander Alekseev
On Wed, Oct 30, 2024 at 5:26 PM Peter Geoghegan <pg@bowt.ie> wrote: > I use Debian unstable for most of my day to day work. Apparently > Debian unstable has exactly the same version of lcov as Ubuntu 24.04. > > I've also been unable to generate coverage reports for some time (at > least on Debian, with LCOV version 2.0-1). I found a temporary workaround. I'm now once again able to produce html coverage reports on my Debian unstable workstation. Here's what I did: I found that a local installation of lcov 1.16 worked. I cloned the lcov git repo, and checked out the tag 'v1.16'. I then installed this older lcov version to a directory under my home directory. It was also necessary to convince the build system to use my known-good version of lcov. I had to use autoconf here, since (if I'm not mistaken) only autoconf will accept specific instructions as to which gcov, lcov, and genhtml to use (via environment variables). I also used a relatively old version of GCC, taken from the "gcc-11" package (not sure if that part was really necessary). I adding something along these lines to the zsh function that runs "configure" for me, to automate the process going forward: export CC="gcc-11" # Provided by gcc-11 package export GCOV="gcov-11" # Provided by gcc-11 package export LCOV="/home/pg/.../lcov" # Provided by local lcov 1.16 installation export GENHTML="/home/pg/.../genhtml" # Provided by local lcov 1.16 installation From here I ran the standard, documented procedure: I ran configure (through the zsh function), built Postgres in the usual way, ran the tests in the usual way, and finally ran "make coverage-html". The final html report looks very similar to the one from coverage.postgresql.org. (There were some geninfo warnings about certain files not having any coverage whatsoever, but I'm pretty sure that that's normal.) -- Peter Geoghegan
Peter Geoghegan <pg@bowt.ie> writes:
> On Wed, Oct 30, 2024 at 5:26 PM Peter Geoghegan <pg@bowt.ie> wrote:
>> I've also been unable to generate coverage reports for some time (at
>> least on Debian, with LCOV version 2.0-1).
> I found a temporary workaround. I'm now once again able to produce
> html coverage reports on my Debian unstable workstation.
I got around to poking at this on a shiny new Fedora 41 image,
with gcc-14.2.1 and lcov-2.0-4. I see errors and warnings aplenty
there too, but after reading the suppression hints that appear in
the error messages I was able to make them all go away:
$ make -s coverage-html GENHTML_FLAGS="-q --legend --ignore-errors unmapped,unmapped"
Overall coverage rate:
lines......: 60.7% (282865 of 466310 lines)
functions......: 67.4% (16114 of 23895 functions)
$
The "-q --legend" part is our default value of GENHTML_FLAGS,
the new magic is the --ignore-errors bit. As far as I can tell,
specifying "unmapped" once reduces that class of errors to
warnings and specifying it again silences them entirely.
With this, I got plausible-looking html output (I didn't vet it
in detail, but a couple of spot checks looked sane).
I'm curious whether a similar workaround will help with the
Debian toolchain.
regards, tom lane
On Wed, Jan 15, 2025 at 4:44 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> With this, I got plausible-looking html output (I didn't vet it
> in detail, but a couple of spot checks looked sane).
>
> I'm curious whether a similar workaround will help with the
> Debian toolchain.
Looks like it will.
I was able to get the lcov git master branch's tip to produce a
Postgres coverage report (same compiler version and OS as before).
Unfortunately, I had to use an even grottier incantation to get this
to work:
make -s coverage-html GENHTML_FLAGS="-q --legend --ignore-errors
unmapped,unmapped,empty,empty,inconsistent,inconsistent,corrupt,corrupt,range,range"
LCOVFLAGS="--ignore-errors
empty,empty,negative,negative,inconsistent,inconsistent"
(As you can imagine, I came up with this through pure trial and error.)
The final report seems fine, though -- at least on a quick look. The
number of times that we actually relied on the suppressions seems to
have been fairly limited, in the end:
Message summary:
11 ignore messages:
empty: 1
inconsistent: 3
negative: 7
Overall coverage rate:
source files: 1527
lines.......: 78.7% (396815 of 504471 lines)
functions...: 86.3% (23710 of 27474 functions)
Note also: I had to install gcov to /usr/local this time. I couldn't
get away with installing it to my home directory (something about Perl
not being able to see lcovutil.pm in @INC).
--
Peter Geoghegan