[Toybox] test flakiness sitrep

enh enh at google.com
Mon Aug 12 13:33:48 PDT 2019


last week i turned on the toybox tests in presubmit for changes to
toybox in AOSP, and postsubmit otherwise. so we have a fair bit of
data now (sadly no statistics because non-gtest tests aren't that well
integrated yet).

bad news:
* i can't merge my own find(1) fix because the test i added is so
flaky i can't get it in. (but really, that's a feature, and should
count as good news.)

good news:
* very little flakiness so far. i've only seen a couple of tests (one
cpio, one tar) failing, and both make me suspicious that there's still
some bad behavior in dd somewhere. (i haven't had time to investigate
at all yet, but i can say that i haven't seen any failures running
these tests on the desktop with GNU dd. so, yes, i also did this to
myself.)

but that's not bad, i reckon. and even the flake is helping find bugs
in dd, it seems :-)


More information about the Toybox mailing list