[Toybox] LLVM sanitizers

[Andy Chu]

(Forking this thread since it got buried ...)

>> I actually ran it under LLVM
>> sanitizers (ASAN/MSAN/etc.), just as I did for toybox, and it revealed
>> the expected C coding bugs, in this code being maintained by one
>> person for 30 years... (BTW you never responded to my last message
>> about that)
>
> My laptop rebooted during txlf and I lost my open windows. I have a todo
> item to look at your test suite suggestions, but when I glanced at the
> start of it, it was things like adding "only run these tests as root"
> guards to some files which are part of any testing triage, so I just
> started doing test suite triage until I ran out of time that day, and
> haven't gotten back to it yet...

The original patches are here:
http://lists.landley.net/pipermail/toybox-landley.net/2016-March/008147.html

Follow up which you didn't reply to:
http://lists.landley.net/pipermail/toybox-landley.net/2016-May/008358.html

> You said the sed -f thing was "literally the first thing you tried" and
> was found with a fuzzer.

Yes, fuzzing and ASAN are separate but related.  I sent patches for
ASAN build/test and ASAN *fixes*, but never sent patches for fuzzing
build/test -- only 1 fuzzing fix.


> The other thing you found outside of pending was commit c73947814aab
> (which was a thinko on my part, I was trusting the -1/2 to be zero, but
> was testing <= not = so it still went through the loop body then), which
> I can't find your submission email for (might have been on irc?) so I
> dunno how you found it.
>
> The other stuff you patched was in pending so hadn't BEEN reviewed.

Not sure how this is relevant... you want both manual review and
automated tools.  Even if you could manually review everything, which
you can't, it doesn't mean that tools won't help you.

Thank you for writing up the June release notes in detail... though I
would say it's inaccurate to say I "noticed" any bugs.  I ran tools
and they showed me where the bugs were.  And then I sent patches to
add them to the build system so other people can do the same.

There are still bugs in the tree that the tools found -- I think I've
been clear about that.  I thought you would treat them with more
urgency.

> Feel free to run it. I've never had much interest in false positive
> generators myself.

This is the same confusion you had before, and I've also pointed this
out before.  The sanitizers are NOT static analysis -- they give you
instrumentated binaries which you run with real input.  They don't
give false positives -- if they flag something like an out of bounds
memory access, then IT HAPPENED.

That's why you need a test suite to use them.

Andy