[Toybox] Release 0.8.10

[enh]

On Tue, Aug 1, 2023 at 5:56 PM enh <enh at google.com> wrote:
>
> On Mon, Jul 31, 2023 at 6:52 PM Rob Landley <rob at landley.net> wrote:
> >
> > On 7/31/23 09:31, enh wrote:
> > > seems like this release is in a pretty bad state?
> >
> > The tests passed locally! All of 'em! With glibc and musl! Sigh...
> >
> > > github CI is failing for both
> > > linux and macOS... linux seems to have some tar failures
> >
> > Yeah, it's those darn sparse failures again. On ext4 writing a sparse file
> > behaves determinstically-ish, but butterfly-effect-fs not so much.
>
> yeah, that's one thing that's really weird --- sometimes the tests
> pass in github's CI anyway.
>
> > Admittedly it's only user visible if you _ask_ for it, and I'm kind of tempted
> > to teach tar that "--sparse means any sufficient run of zeroes becomes #*%(#&
> > sparse whether or not the filesystem knows about it". Where "sufficient" would
> > logically be 512 byte aligned 512 byte blocks, because that's how tar thinks.
> > (It's a space savings. I don't THINK there's a maximum number of sparse extents?
> > I've even got a realloc every 512 entries in the existing loop! And a note to
> > figure out how to test that properly. Don't ask me what gnu/dammit does with a
> > multiblock sparse table, it _probably_ works?)
> >
> > *shrug* If nothing else it would eliminate the filesystem dependency...
> >
> > > FAIL: tar sparse without overflow
> > > echo -ne '' | tar c --owner root --group sys --mtime @1234567890 --sparse fweep
> > > | SUM 3
> > > --- expected 2023-07-29 01:27:20.471064281 +0000
> > > +++ actual 2023-07-29 01:27:20.475064343 +0000
> > > @@ -1 +1 @@
> > > -50dc56c3c7eed163f0f37c0cfc2562852a612ad0
> > > +4b0cf135987e8330d8a62433471faddccfabac75
> >
> > In order for this to be happening the sparse test I added at the start has to
> > pass, but then the larger saving-of-sparseness does not match the file we just
> > created on the previous line.
> >
> > I.E. the microsoft github behavior has to be INCONSISTENT within the same run to
> > trigger this. Wheee...
> >
> > > and a mv failure
> > >
> > > FAIL: mv random file to new file
> > > echo -ne '' | mv file1 file2 && [ ! -e file1 -a -f file2 ] && stat -c %s file2
> > > --- expected 2023-07-29 01:27:12.202935388 +0000
> > > +++ actual 2023-07-29 01:27:12.206935451 +0000
> > > @@ -1 +1 @@
> > > -5243392
> > > +10752
> > >
> > > though the latter might be flaky?
> >
> > It's another sparse file:
> >
> > dd if=/dev/zero of=file1 seek=10k count=1 >/dev/null 2>&1
> >
> > And this one is even WEIRDER because stat %s is "total size in bytes" which
> > toybox gets from stat->st_size which man 2 stat says is:
> >
> >                off_t     st_size;        /* Total size, in bytes */
> >
> > Which is NOT the st_blocks field, this is the ls -l size.
> >
> > I thought maybe my new dd went weird but I just ran "ASAN=1 make clean defconfig
> > toybox tests" here _again_ and it passed? Short reads/writes should be
> > compensated for (it multiples up a total count of bytes and then tries to shovel
> > them in ifs and ofs sized chunks but the _total_ still has to add up).
> >
> > 10752 is 10*1024+512 meaning the seek was in done in BYTES not blocks? But
> > that's not what my dd is doing:
> >
> > $ ./dd if=/dev/zero of=file1 seek=10k count=1
> > 1+0 records in
> > 1+0 records out
> > 512 bytes (512 B) copied, 0.001 s, 500 K/s
> > $ ls -l file1
> > -rw-r--r-- 1 landley landley 5243392 Jul 31 20:19 file1
> >
> > And looking at the code the only way it _would_ do that is...
> >
> > Oh. Duh. I moved TT.iflag from globals (where it's guaranteed zeroed) to a local
> > variable (which needs manual zeroing), and of course despite gcc regularly
> > producing "is provably never used uninitialized" warnings, it does NOT warn for
> > ACTUALLY UNINITIALIZED VARIABLES when you pass their address to an inlineable
> > function in the same translation unit.
> >
> > Yup, my bad. Bug in the new dd introduced by my cleanup last week. Uninitialized
> > variable hallucinating iflag=skip_bytes. Commit a88bd5e9a0c5.
> >
> > (Grumble grumble false sense of security. And you'd think ASAN would
> > randomize/poison that sort of thing. Memset new stack pages with 0x55 or some
> > such...)
> >
> > > i tried to reproduce locally, but hit a few different asan failures. linux has
> > > an asan failure in blkid (patch sent already) and macOS dies in the id 0 test:
> >
> > The downside of testing glibc+gcc's asan in debian. (Still can't use the NDK one
> > because it only ships asan.so not asan.a.)
> >
> > > looks like a classic "the passwd functions don't guarantee the lifetime of a
> > > result past another call to a passwd function" issue?
> >
> > Try commit 3d86ee9eaec1?
>
> (yeah, github CI confirms that works too.)
>
> > Which works on both glibc and musl, with ASAN on the glibc build and when I
> > enable ASAN on the musl build the cross compiler goes "x86_64-linux-musl-cc:
> > fatal error: cannot read spec file 'libsanitizer.spec': No such file or
> > directory" so that's nice...
> >
> > > linux also dies in the sed timeout test; that seems to be a pathological case
> > > for asan because increasing the timeout to 60s also didn't pass. (though
> > > weirdly, that test is fine -- finishing almost instantly, just like non-asan
> > > -- on macOS.
> >
> > Didn't see it on debian's gcc+glibc ASAN, but mostly likely that has fewer checks.
>
> (to be fair, i actually have no idea of the state of the gcc asan; but
> all the people _i_ know who work on asan-type stuff for a living work
> on the llvm one.)
>
> > > not sure whether that's a bsd/glibc difference or a linux-only asan
> > > bug. the latter seems less likely, but i'll mention it to the asan folks anyway...)
> >
> > I remind you of:
> >
> > commit c0dca293c1301a6315684703706597db07a8dbe1
> > Author: Rob Landley <rob at landley.net>
> > Date:   Sat Jun 27 03:14:49 2020 -0500
> >
> >     The bionic/clang asan plumbing slows the test down >10x, so expand timeout.
> >
> > That test is ping-ponging between a bunch of different segments (the source
> > buffer, the destination buffer, the parsed regex struct, and the stack, global
> > variables, the toybox text segment, and glibc's library text segment) and it's
> > entirely possible whatever virtual TLB setup ASAN does to catch weirdness is
> > getting thrashed. Worse now than when the 20 second timeout was enough...
>
> /me wonders if the reason i think this is fine "on macOS" is because i
> actually mean "on an M1 because it has truly insane memory bandwidth
> [at the cost of non-upgradeable memory, of course]".
>
> i'd report the results of running the asan tests on an x86-64 mac here
> ... except that it crashes immediately the first time it starts
> toybox, somewhere deep in libclang_rt.asan_osx_dynamic.dylib (their
> fault, not yours).
>
> but, yeah, my M1 mac is passing everything quickly right now.
>
> > Meanwhile, without ASAN wrapping date +%s.%N around the test says it takes a
> > quarter of a second on my 10 year old laptop:
> >
> > 1690850299.108588387
> > PASS: sed megabyte s/x/y/g (20 sec timeout)
> > 1690850299.342395058
> >
> > A reasonable chunk of which is the shell test plumbing. (Just two consecutive
> > "date +%s.%N; date+%s.%N" calls from the shell are .007 seconds apart on this
> > machine, nontrivial chunk of that 250 milliseconds. I think the original 10
> > second timeout was to make it reliably pass on my 66mhz Turtle board.)
> >
> > I can't think of a fix here other than disabling the test...
>
> yeah, or skipping if $ASAN is set? :-(
>
> for now, though, Android's CI doesn't care as long as *hwasan* is fast
> enough, and a quick test on an aosp_cheetah_hwasan-userdebug device
> says ... "can't create /expected: Read-only file system". oh. hmm.
> looks like https://github.com/landley/toybox/commit/03e1cc1e45b67ad65e5ad0ae47b7a54e68d929d5
> broke things. not sure why $TESTDIR isn't set for me? oh, because
> that's set by scripts/test.sh which we don't use --- we call
> scripts/runtest.sh directly.
>
> too late for that to be today's problem though... i'll look further tomorrow!
>
> ah, fuck it, i'll only spend the evening wondering...
>
> yes, with the obvious line added to run-tests-on-android.sh, all the
> tests pass on my hwasan build (and the sed test only takes a couple of
> seconds). (for reference, my linux/x86-64 hardware that timed out was
> a work amd threadripper box, not my personal 10 year old laptop!)

hmm... missed one failure. my idea of what $TESTDIR is meant to be and
yours obviously not exactly the same...

-- ls
FAIL: ls no argument
echo -ne '' | "/system/bin/ls"
--- expected 2023-08-02 00:40:11.231545936 +0000
+++ actual 2023-08-02 00:40:11.327545936 +0000
@@ -1,4 +1,6 @@
+actual
 dir1
 dir2
+expected
 file1.txt
 file2.txt


> > > i *couldn't* reproduce the tar failures locally (the failure at the top was copy
> > > & pasted from github ci results), so maybe another lurking fs dependency?
> >
> > A fs dependency which commit d69584882ff8 worked around for five minutes.
> >
> > Rob
> >
> > P.S. Part of my drive to run tests in the known mkroot environment is although I
> > do want to catch all the real world weirdness, this is no good for REGRESSION
> > testing because "distro du jour" that runs the test suite for the first time has
> > failures that are due to assumptions IN THE TESTS rather than anything toybox
> > did wrong per se. Writing robust portable tests that actually SAY anything is
> > REALLY HARD. There's a zillion tests I haven't even tried to write yet because
> > "I could get this to pass once" is NOT something you put in the test suite.