[Toybox] generating toybox test coverage data
rob at landley.net
Wed Jun 17 14:06:45 PDT 2020
On 6/16/20 5:41 PM, enh via Toybox wrote:
> TL;DR: the current state of test coverage looks like:
> function: 59.85% (468/782)
> line: 56.17% (12111/21560)
> region: 53.61% (11280/21039)
See "test error paths", better coverage for existing tests, adding tests for
commands that have no tests...
When people come asking me how they can help I never know what to say, and for a
while I tried pointing at the test suite, and this resulted in tests that test
the kernel rather than testing toybox. (Hence
https://github.com/landley/toybox/commit/d6f8c41e2542 which I still need to
Turns out there's design work in testing too.
> but in case anyone's interested in actually reproducing my results:
> cd /tmp
> git clone https://github.com/landley/toybox.git toybox-coverage
> cd toybox-coverage
> export CC=clang
> export CFLAGS="-fprofile-instr-generate -fcoverage-mapping"
> export LDFLAGS="-fprofile-instr-generate -fcoverage-mapping"
> apply this patch because atexit() doesn't work in toybox:
Try sigatexit(), and the answer to why is "error recovery in shell builtins".
(What's the portable way to _remove_ a libc function from the atexit() list?)
> make defconfig
> export LLVM_PROFILE_FILE="/tmp/coverage/%p"
> TEST_ALL=1 make tests
> llvm-profdata-9 merge -sparse /tmp/coverage/* -o toybox.profdata
> llvm-cov-9 show ./toybox -instr-profile=toybox.profdata
> -show-instantiations=false -format html -output-dir=html-coverage
> google-chrome ./html-coverage/index.html
If "make bloatcheck" is a thing, possibly this should have a make tests_coverage
More information about the Toybox