[Toybox] generating toybox test coverage data

[Rob Landley]

On 6/16/20 5:41 PM, enh via Toybox wrote:
> TL;DR: the current state of test coverage looks like:
> 
> function:  59.85% (468/782)
> line:  56.17% (12111/21560)
> region:  53.61% (11280/21039)

See "test error paths", better coverage for existing tests, adding tests for
commands that have no tests...

When people come asking me how they can help I never know what to say, and for a
while I tried pointing at the test suite, and this resulted in tests that test
the kernel rather than testing toybox. (Hence
https://github.com/landley/toybox/commit/d6f8c41e2542 which I still need to
revisit...)

Turns out there's design work in testing too.

> but in case anyone's interested in actually reproducing my results:
> 
> cd /tmp
> git clone https://github.com/landley/toybox.git toybox-coverage
> cd toybox-coverage
> export CC=clang
> export CFLAGS="-fprofile-instr-generate -fcoverage-mapping"
> export LDFLAGS="-fprofile-instr-generate -fcoverage-mapping"
> 
> apply this patch because atexit() doesn't work in toybox:

Try sigatexit(), and the answer to why is "error recovery in shell builtins".
(What's the portable way to _remove_ a libc function from the atexit() list?)

> make defconfig
> make
> export LLVM_PROFILE_FILE="/tmp/coverage/%p"
> TEST_ALL=1 make tests
> llvm-profdata-9 merge -sparse /tmp/coverage/* -o toybox.profdata
> llvm-cov-9 show ./toybox -instr-profile=toybox.profdata
> -show-instantiations=false -format html -output-dir=html-coverage
> google-chrome ./html-coverage/index.html

If "make bloatcheck" is a thing, possibly this should have a make tests_coverage
target...

Rob