[Toybox] [PATCH] Fix truncate.test for macOS.

[enh]

On Mon, Jul 4, 2022 at 10:38 PM Rob Landley <rob at landley.net> wrote:

> On 7/4/22 13:08, enh wrote:> On Mon, Jul 4, 2022 at 3:14 AM Rob Landley
> <rob at landley.net> wrote:
> >     On 6/27/22 14:40, enh wrote:
> >     > On Sat, Jun 25, 2022 at 3:45 PM Rob Landley <rob at landley.net
> >     >     >     The problem with the mac tar test is even though it's
> easy enough to
> >     >     find what
> >     >     >     /etc/passwd calls UID 0:
> >     >     >
> >     >     >     ROOT="$(sed -n '/[^:]*:[^:]*:0:/s/:.*//p' /etc/passwd)"
> >     >     >
> >     >     >     That doesn't change the fact it'll be putting a
> different string
> >     into the
> >     >     >     tarball, with different sha1sums. Um. (I was using
> "root" as the
> >     one known
> >     >     >     constant account that didn't vary across distros.
> Possibly I need a
> >     >     way to tell
> >     >     >     it to use an alternate /etc/passwd file to lookup
> usernames. This is
> >     >     why I've
> >     >     >     been poking at mkroot, but making that work on a mac is
> just... ow.)
> >     >
> >     >     FYI, I committed your patch shortly after sending that message.
> >     >
> >     > thanks. interestingly, i realized that i think we also wouldn't
> get a red
> >     cross
> >     > in the github ui if we broke the _linux_ tests?
> >
> >     Hmmm... looks like I broke VERBOSE=all's exit code when I moved the
> tests into a
> >     subshell (commit e00b4c26553b) and de474ba03950 wasn't a full enough
> fix...
> >
> >     Try now?
> >
> >
> > looks plausible: github's saying that linux passed
> > (
> https://github.com/landley/toybox/runs/7178532773?check_suite_focus=true)
> but
> > macos failed
> > (
> https://github.com/landley/toybox/runs/7178532509?check_suite_focus=true).
>
> Probably accurate.
>
> > (btw, one thing i did with the Android toybox test runner was collect the
> > _names_ of the failing tests. that's a handy thing to have at the bottom
> of the
> > output, but perhaps more handy for me when i have to answer the question
> "is
> > this bug a dupe of the existing one i'm already looking at, or do i have
> two
> > problems?".)
>
> On current devuan TEST_HOST:
>
> $ TEST_HOST=1 VERBOSE=allnopass make tests 2>/dev/null | grep ^FAIL:
> FAIL: cpio skip NUL
> FAIL: cut -bO overlaps
> FAIL: cut -DF
> FAIL: cut -DF 2
> FAIL: factor -h
> FAIL: factor -x
> FAIL: hexdump simple file
> FAIL: hexdump simple file -b
> FAIL: hexdump simple file -c
> FAIL: hexdump simple file -d
> FAIL: hexdump simple file -o
> FAIL: hexdump simple file -x
> FAIL: hexdump head of file -n 10
> FAIL: hexdump skip head of file -s 10
> FAIL: hexdump squeeze repeating lines
> FAIL: hexdump don't squeeze repeating lines
> FAIL: hexdump accumulate offset accross files
> FAIL: iconv chars | xxd -p
> FAIL: iconv -t UTF-16BE
> FAIL: iconv -t UTF-16LE
> FAIL: iconv -t UTF-32BE
> FAIL: iconv -t UTF-32LE
> FAIL: netcat more than buffer bytes left at end
> FAIL: tac file1 notfound file2
> FAIL: top -d invalid input
> FAIL: top -d unknown suffix
> FAIL: top -d suffix trailing junk
> FAIL: unzip -l
> FAIL: unzip -lq
> FAIL: unzip -lv
> FAIL: unzip -v
> FAIL: wc one file
> FAIL: wc -m
> FAIL: wc -m 2
> FAIL: wc -mlw
>
> Which is because upgrading from devuan banana to devuan cantaloupe randomly
> changed the behavior of a zillion gnu/dammit tools and I haven't gone
> through
> and fixed it yet...
>
> I hadn't bothered having it collect that and print a summary report
> because the
> output's easily filterable...?
>
> >     Is there a portable way to determine filesystem type, though? df .
> doesn't say,
> >     I have to look in /proc/mounts and I doubt mac has that?
> >
> >     $ grep -w "^$(df . | tail -n 1 | toybox cut -DF 1)" /proc/mounts |
> toybox
> >     cut -DF 3
> >
> >     (I THINK that if the device has a space in it df will output the
> escaped
> >     version, which should match for grep... But again: Linux.)
> >
> > on the mac it looks like mount is your best bet:
> >
> > ~$ mount
> > /dev/disk3s1s1 on / (apfs, sealed, local, read-only, journaled)
> > ...
> >
> > but maybe just "it's a mac, it's probably using apple's default fs" is
> sufficient?
>
> Hmmm...
>
> sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
>
> Word 4 vs word 5. I guess I could check for "type" and advance, else strip
> off
> the starting "("... Of course now that I've sat down to implement an
> is_filesystem shell function, I'm trying to figure out what the right
> question
> to ask here, and checking for a single filesystem type is basically
> is_mkroot,
> while a whitelist of known filesystem types is bound to accordion out
> unpleasantly while STILL breaking for people...
>
> Blah. This is a design issue. What behavior do we WANT here?
>
> Query: what filesystem A) are you running this under on the android test
> system,
> B) is available as "scratch space" to an application on an android phone?
>

depends on the device. there are some restrictions (and actually the
writeable space might be (a) more restricted and (b) less likely something
that the OEMs change) but since the toybox tests aren't (yet) in CTS,
"works on Google's CI" is good enough for now. we can worry about "how do
these tests work on every device configuration that it's possible to ship?"
if/when that's an actual problem.


> We never did revisit the "posix container app" idea. One that could have a
> local
> root within its container, and access to maybe 10 UID/GIDs. After getting
> the
> tests running in mkroot, I want to get the tests running on an Android
> device/VM...
>
> (In mkroot, I'm probably using either tmpfs or ext2. If one of the other
> environments were using ext3 or ext4 I could be convinced to advance the
> version
> there, and similarly could layer unionfs on top if it was already being
> used in
> somebody else's environment, but otherwise would probably stick to the
> simplest
> thing. I note I wrote most of a mke2fs once and ext3 support was easy to
> add,
> but ext4 looks like a can 'o tree-structured worms. I've also got half a
> vfat
> formatter written, but testing on vfat would be creepy and I'm not sure
> what it
> would prove anyway. The other filesystems I've poked at doing creation
> plumbing
> for are isofs and squashfs, but they're not as far along...)
>
> >     >     Another TODO item is packing up debootstrap and alpine root
> >     filesystems to test
> >     >     under mkroot as more rigorous "TEST_HOST=1" runs. With the
> kernel
> >     >     config/version, mountpoint selection, and qemu board emulation
> parameters.
> >     >     Presumably running my init script instead of theirs to do the
> setup
> >     and start
> >     >     the test, but using their $PATH of binaries (gnu/fsf and
> busybox,
> >     respectively).
> >     >     But that's after I get the base mkroot testing well...
> >     >
> >     >     > note that it's /etc/*group* that's weird, not *passwd*. uid
> 0 is
> >     root, but
> >     >     group
> >     >     > 0 is wheel. (i think that's true of all bsds?)
> >
> >     I suspect the easy solution is "skipnot grep -qw root /etc/group".
> >
> >     Because it's not gonna make the same tarball otherwise, and this is
> testing data
> >     fetched out of /etc/passwd. (We have --owner=NAME:ID tests: this
> isn't that.)
> >
> > wouldn't it make more sense to use one of the low-numbered groups that's
> the
> > same on both though, rather than skip?
>
> Grrr... trusting "root" to be there seemed as universal as you could get.
> Slowaris had "root". AIX had "root". I very vaguely recall Eunice on the
> VAX
> _thinking_ it had "root" although I was something like thirteen years old
> at the
> time so it's anybody's guess whether that's accurate. (I don't remember
> what my
> Uncle Alan's HP minicomputer in his den had when I was 7, I was just
> playing
> hangman on the teletype. Yes, actual printer.)
>
> Then again, I probably only remember looking at the USER root, who paid
> attention to groups after 1980? Having the user and group gratuitously
> different
> (but only for this ONE user, presumably daemon/daemon and bin/bin and so on
> match) is... a historical bug in BSD? As with having "eth0" not exist in
> /dev
> when everything ELSE is a file: turns out Bill Joy did not quite grok this
> unix
> thing...
>
> Anyway, I've got this todo item in the tar --transform bucket, lemme
> finish up
> with diff first. (And file off another couple rough edges on sh I've got
> in-progress...)
>
> >     The second truncate makes the file NOT SPARSE.
> >
> >     > the weird part for me was that it wasn't obvious _what_ the
> non-zero
> >     number was
> >     > going to be.
> >
> >     The bug seems to be if you extend a sparse file the result is not
> sparse. I'm
> >     guessing if you go:
> >
> >     truncate -s 1k empty; truncate -s 2m empty; stat -c %b empty
> >
> >     You'll get 4096. (Because 2048*512=1m and that's what we asked it to
> expand the
> >     sparse file to.)
> >
> >
> > /tmp$ ~/toybox/toybox truncate -s 1k empty; ~/toybox/toybox truncate -s
> 2m
> > empty; ~/toybox/toybox stat -c %b empty
> > 4096
>
> Yup. Looks like a mac kernel bug to me. You can extend an empty file
> sparsely,
> and can extend a non-sparse file sparsely, but if you extend a sparse file
> it
> allocates all the zeroes.
>
> >     >     > iirc it's a bit simpler than that (if you don't have some
> company
> >     policy that
> >     >     > says you can only install binaries from their servers) ---
> you just
> >     run "make"
> >     >     > and it pops up a window saying "you want to install all that
> shit?" and
> >     >     you say
> >     >     > "it's a unix system; i know this", bish bash bosh, job done.
>
> Do you actually have to say the line from Jurassic park out loud? Or do
> you type
> it at the script?
>

Apple's not that cool ... you just _think_ that while clicking "okay" :-)


> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20220725/46e5695d/attachment-0001.htm>