[Toybox] [PATCH] Fix truncate.test for macOS.

[Rob Landley]

On 7/4/22 13:08, enh wrote:> On Mon, Jul 4, 2022 at 3:14 AM Rob Landley
<rob at landley.net> wrote:
>     On 6/27/22 14:40, enh wrote:
>     > On Sat, Jun 25, 2022 at 3:45 PM Rob Landley <rob at landley.net
>     >     >     The problem with the mac tar test is even though it's easy enough to
>     >     find what
>     >     >     /etc/passwd calls UID 0:
>     >     >
>     >     >     ROOT="$(sed -n '/[^:]*:[^:]*:0:/s/:.*//p' /etc/passwd)"
>     >     >
>     >     >     That doesn't change the fact it'll be putting a different string
>     into the
>     >     >     tarball, with different sha1sums. Um. (I was using "root" as the
>     one known
>     >     >     constant account that didn't vary across distros. Possibly I need a
>     >     way to tell
>     >     >     it to use an alternate /etc/passwd file to lookup usernames. This is
>     >     why I've
>     >     >     been poking at mkroot, but making that work on a mac is just... ow.)
>     >
>     >     FYI, I committed your patch shortly after sending that message.
>     >
>     > thanks. interestingly, i realized that i think we also wouldn't get a red
>     cross
>     > in the github ui if we broke the _linux_ tests?
> 
>     Hmmm... looks like I broke VERBOSE=all's exit code when I moved the tests into a
>     subshell (commit e00b4c26553b) and de474ba03950 wasn't a full enough fix...
> 
>     Try now?
> 
> 
> looks plausible: github's saying that linux passed
> (https://github.com/landley/toybox/runs/7178532773?check_suite_focus=true) but
> macos failed
> (https://github.com/landley/toybox/runs/7178532509?check_suite_focus=true).

Probably accurate.

> (btw, one thing i did with the Android toybox test runner was collect the
> _names_ of the failing tests. that's a handy thing to have at the bottom of the
> output, but perhaps more handy for me when i have to answer the question "is
> this bug a dupe of the existing one i'm already looking at, or do i have two
> problems?".)

On current devuan TEST_HOST:

$ TEST_HOST=1 VERBOSE=allnopass make tests 2>/dev/null | grep ^FAIL:
FAIL: cpio skip NUL
FAIL: cut -bO overlaps
FAIL: cut -DF
FAIL: cut -DF 2
FAIL: factor -h
FAIL: factor -x
FAIL: hexdump simple file
FAIL: hexdump simple file -b
FAIL: hexdump simple file -c
FAIL: hexdump simple file -d
FAIL: hexdump simple file -o
FAIL: hexdump simple file -x
FAIL: hexdump head of file -n 10
FAIL: hexdump skip head of file -s 10
FAIL: hexdump squeeze repeating lines
FAIL: hexdump don't squeeze repeating lines
FAIL: hexdump accumulate offset accross files
FAIL: iconv chars | xxd -p
FAIL: iconv -t UTF-16BE
FAIL: iconv -t UTF-16LE
FAIL: iconv -t UTF-32BE
FAIL: iconv -t UTF-32LE
FAIL: netcat more than buffer bytes left at end
FAIL: tac file1 notfound file2
FAIL: top -d invalid input
FAIL: top -d unknown suffix
FAIL: top -d suffix trailing junk
FAIL: unzip -l
FAIL: unzip -lq
FAIL: unzip -lv
FAIL: unzip -v
FAIL: wc one file
FAIL: wc -m
FAIL: wc -m 2
FAIL: wc -mlw

Which is because upgrading from devuan banana to devuan cantaloupe randomly
changed the behavior of a zillion gnu/dammit tools and I haven't gone through
and fixed it yet...

I hadn't bothered having it collect that and print a summary report because the
output's easily filterable...?

>     Is there a portable way to determine filesystem type, though? df . doesn't say,
>     I have to look in /proc/mounts and I doubt mac has that?
> 
>     $ grep -w "^$(df . | tail -n 1 | toybox cut -DF 1)" /proc/mounts | toybox
>     cut -DF 3
> 
>     (I THINK that if the device has a space in it df will output the escaped
>     version, which should match for grep... But again: Linux.)
> 
> on the mac it looks like mount is your best bet:
> 
> ~$ mount
> /dev/disk3s1s1 on / (apfs, sealed, local, read-only, journaled)
> ...
> 
> but maybe just "it's a mac, it's probably using apple's default fs" is sufficient?

Hmmm...

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)

Word 4 vs word 5. I guess I could check for "type" and advance, else strip off
the starting "("... Of course now that I've sat down to implement an
is_filesystem shell function, I'm trying to figure out what the right question
to ask here, and checking for a single filesystem type is basically is_mkroot,
while a whitelist of known filesystem types is bound to accordion out
unpleasantly while STILL breaking for people...

Blah. This is a design issue. What behavior do we WANT here?

Query: what filesystem A) are you running this under on the android test system,
B) is available as "scratch space" to an application on an android phone?

We never did revisit the "posix container app" idea. One that could have a local
root within its container, and access to maybe 10 UID/GIDs. After getting the
tests running in mkroot, I want to get the tests running on an Android device/VM...

(In mkroot, I'm probably using either tmpfs or ext2. If one of the other
environments were using ext3 or ext4 I could be convinced to advance the version
there, and similarly could layer unionfs on top if it was already being used in
somebody else's environment, but otherwise would probably stick to the simplest
thing. I note I wrote most of a mke2fs once and ext3 support was easy to add,
but ext4 looks like a can 'o tree-structured worms. I've also got half a vfat
formatter written, but testing on vfat would be creepy and I'm not sure what it
would prove anyway. The other filesystems I've poked at doing creation plumbing
for are isofs and squashfs, but they're not as far along...)

>     >     Another TODO item is packing up debootstrap and alpine root
>     filesystems to test
>     >     under mkroot as more rigorous "TEST_HOST=1" runs. With the kernel
>     >     config/version, mountpoint selection, and qemu board emulation parameters.
>     >     Presumably running my init script instead of theirs to do the setup
>     and start
>     >     the test, but using their $PATH of binaries (gnu/fsf and busybox,
>     respectively).
>     >     But that's after I get the base mkroot testing well...
>     >
>     >     > note that it's /etc/*group* that's weird, not *passwd*. uid 0 is
>     root, but
>     >     group
>     >     > 0 is wheel. (i think that's true of all bsds?)
> 
>     I suspect the easy solution is "skipnot grep -qw root /etc/group".
> 
>     Because it's not gonna make the same tarball otherwise, and this is testing data
>     fetched out of /etc/passwd. (We have --owner=NAME:ID tests: this isn't that.)
> 
> wouldn't it make more sense to use one of the low-numbered groups that's the
> same on both though, rather than skip?

Grrr... trusting "root" to be there seemed as universal as you could get.
Slowaris had "root". AIX had "root". I very vaguely recall Eunice on the VAX
_thinking_ it had "root" although I was something like thirteen years old at the
time so it's anybody's guess whether that's accurate. (I don't remember what my
Uncle Alan's HP minicomputer in his den had when I was 7, I was just playing
hangman on the teletype. Yes, actual printer.)

Then again, I probably only remember looking at the USER root, who paid
attention to groups after 1980? Having the user and group gratuitously different
(but only for this ONE user, presumably daemon/daemon and bin/bin and so on
match) is... a historical bug in BSD? As with having "eth0" not exist in /dev
when everything ELSE is a file: turns out Bill Joy did not quite grok this unix
thing...

Anyway, I've got this todo item in the tar --transform bucket, lemme finish up
with diff first. (And file off another couple rough edges on sh I've got
in-progress...)

>     The second truncate makes the file NOT SPARSE.
> 
>     > the weird part for me was that it wasn't obvious _what_ the non-zero
>     number was
>     > going to be. 
> 
>     The bug seems to be if you extend a sparse file the result is not sparse. I'm
>     guessing if you go:
> 
>     truncate -s 1k empty; truncate -s 2m empty; stat -c %b empty
> 
>     You'll get 4096. (Because 2048*512=1m and that's what we asked it to expand the
>     sparse file to.)
> 
> 
> /tmp$ ~/toybox/toybox truncate -s 1k empty; ~/toybox/toybox truncate -s 2m
> empty; ~/toybox/toybox stat -c %b empty
> 4096

Yup. Looks like a mac kernel bug to me. You can extend an empty file sparsely,
and can extend a non-sparse file sparsely, but if you extend a sparse file it
allocates all the zeroes.

>     >     > iirc it's a bit simpler than that (if you don't have some company
>     policy that
>     >     > says you can only install binaries from their servers) --- you just
>     run "make"
>     >     > and it pops up a window saying "you want to install all that shit?" and
>     >     you say
>     >     > "it's a unix system; i know this", bish bash bosh, job done.

Do you actually have to say the line from Jurassic park out loud? Or do you type
it at the script?

Rob