<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jul 4, 2022 at 10:38 PM Rob Landley <<a href="mailto:rob@landley.net">rob@landley.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 7/4/22 13:08, enh wrote:> On Mon, Jul 4, 2022 at 3:14 AM Rob Landley<br>
<<a href="mailto:rob@landley.net" target="_blank">rob@landley.net</a>> wrote:<br>
> On 6/27/22 14:40, enh wrote:<br>
> > On Sat, Jun 25, 2022 at 3:45 PM Rob Landley <<a href="mailto:rob@landley.net" target="_blank">rob@landley.net</a><br>
> > > The problem with the mac tar test is even though it's easy enough to<br>
> > find what<br>
> > > /etc/passwd calls UID 0:<br>
> > ><br>
> > > ROOT="$(sed -n '/[^:]*:[^:]*:0:/s/:.*//p' /etc/passwd)"<br>
> > ><br>
> > > That doesn't change the fact it'll be putting a different string<br>
> into the<br>
> > > tarball, with different sha1sums. Um. (I was using "root" as the<br>
> one known<br>
> > > constant account that didn't vary across distros. Possibly I need a<br>
> > way to tell<br>
> > > it to use an alternate /etc/passwd file to lookup usernames. This is<br>
> > why I've<br>
> > > been poking at mkroot, but making that work on a mac is just... ow.)<br>
> ><br>
> > FYI, I committed your patch shortly after sending that message.<br>
> ><br>
> > thanks. interestingly, i realized that i think we also wouldn't get a red<br>
> cross<br>
> > in the github ui if we broke the _linux_ tests?<br>
> <br>
> Hmmm... looks like I broke VERBOSE=all's exit code when I moved the tests into a<br>
> subshell (commit e00b4c26553b) and de474ba03950 wasn't a full enough fix...<br>
> <br>
> Try now?<br>
> <br>
> <br>
> looks plausible: github's saying that linux passed<br>
> (<a href="https://github.com/landley/toybox/runs/7178532773?check_suite_focus=true" rel="noreferrer" target="_blank">https://github.com/landley/toybox/runs/7178532773?check_suite_focus=true</a>) but<br>
> macos failed<br>
> (<a href="https://github.com/landley/toybox/runs/7178532509?check_suite_focus=true" rel="noreferrer" target="_blank">https://github.com/landley/toybox/runs/7178532509?check_suite_focus=true</a>).<br>
<br>
Probably accurate.<br>
<br>
> (btw, one thing i did with the Android toybox test runner was collect the<br>
> _names_ of the failing tests. that's a handy thing to have at the bottom of the<br>
> output, but perhaps more handy for me when i have to answer the question "is<br>
> this bug a dupe of the existing one i'm already looking at, or do i have two<br>
> problems?".)<br>
<br>
On current devuan TEST_HOST:<br>
<br>
$ TEST_HOST=1 VERBOSE=allnopass make tests 2>/dev/null | grep ^FAIL:<br>
FAIL: cpio skip NUL<br>
FAIL: cut -bO overlaps<br>
FAIL: cut -DF<br>
FAIL: cut -DF 2<br>
FAIL: factor -h<br>
FAIL: factor -x<br>
FAIL: hexdump simple file<br>
FAIL: hexdump simple file -b<br>
FAIL: hexdump simple file -c<br>
FAIL: hexdump simple file -d<br>
FAIL: hexdump simple file -o<br>
FAIL: hexdump simple file -x<br>
FAIL: hexdump head of file -n 10<br>
FAIL: hexdump skip head of file -s 10<br>
FAIL: hexdump squeeze repeating lines<br>
FAIL: hexdump don't squeeze repeating lines<br>
FAIL: hexdump accumulate offset accross files<br>
FAIL: iconv chars | xxd -p<br>
FAIL: iconv -t UTF-16BE<br>
FAIL: iconv -t UTF-16LE<br>
FAIL: iconv -t UTF-32BE<br>
FAIL: iconv -t UTF-32LE<br>
FAIL: netcat more than buffer bytes left at end<br>
FAIL: tac file1 notfound file2<br>
FAIL: top -d invalid input<br>
FAIL: top -d unknown suffix<br>
FAIL: top -d suffix trailing junk<br>
FAIL: unzip -l<br>
FAIL: unzip -lq<br>
FAIL: unzip -lv<br>
FAIL: unzip -v<br>
FAIL: wc one file<br>
FAIL: wc -m<br>
FAIL: wc -m 2<br>
FAIL: wc -mlw<br>
<br>
Which is because upgrading from devuan banana to devuan cantaloupe randomly<br>
changed the behavior of a zillion gnu/dammit tools and I haven't gone through<br>
and fixed it yet...<br>
<br>
I hadn't bothered having it collect that and print a summary report because the<br>
output's easily filterable...?<br>
<br>
> Is there a portable way to determine filesystem type, though? df . doesn't say,<br>
> I have to look in /proc/mounts and I doubt mac has that?<br>
> <br>
> $ grep -w "^$(df . | tail -n 1 | toybox cut -DF 1)" /proc/mounts | toybox<br>
> cut -DF 3<br>
> <br>
> (I THINK that if the device has a space in it df will output the escaped<br>
> version, which should match for grep... But again: Linux.)<br>
> <br>
> on the mac it looks like mount is your best bet:<br>
> <br>
> ~$ mount<br>
> /dev/disk3s1s1 on / (apfs, sealed, local, read-only, journaled)<br>
> ...<br>
> <br>
> but maybe just "it's a mac, it's probably using apple's default fs" is sufficient?<br>
<br>
Hmmm...<br>
<br>
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)<br>
<br>
Word 4 vs word 5. I guess I could check for "type" and advance, else strip off<br>
the starting "("... Of course now that I've sat down to implement an<br>
is_filesystem shell function, I'm trying to figure out what the right question<br>
to ask here, and checking for a single filesystem type is basically is_mkroot,<br>
while a whitelist of known filesystem types is bound to accordion out<br>
unpleasantly while STILL breaking for people...<br>
<br>
Blah. This is a design issue. What behavior do we WANT here?<br>
<br>
Query: what filesystem A) are you running this under on the android test system,<br>
B) is available as "scratch space" to an application on an android phone?<br></blockquote><div><br></div><div>depends on the device. there are some restrictions (and actually the writeable space might be (a) more restricted and (b) less likely something that the OEMs change) but since the toybox tests aren't (yet) in CTS, "works on Google's CI" is good enough for now. we can worry about "how do these tests work on every device configuration that it's possible to ship?" if/when that's an actual problem.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
We never did revisit the "posix container app" idea. One that could have a local<br>
root within its container, and access to maybe 10 UID/GIDs. After getting the<br>
tests running in mkroot, I want to get the tests running on an Android device/VM...<br>
<br>
(In mkroot, I'm probably using either tmpfs or ext2. If one of the other<br>
environments were using ext3 or ext4 I could be convinced to advance the version<br>
there, and similarly could layer unionfs on top if it was already being used in<br>
somebody else's environment, but otherwise would probably stick to the simplest<br>
thing. I note I wrote most of a mke2fs once and ext3 support was easy to add,<br>
but ext4 looks like a can 'o tree-structured worms. I've also got half a vfat<br>
formatter written, but testing on vfat would be creepy and I'm not sure what it<br>
would prove anyway. The other filesystems I've poked at doing creation plumbing<br>
for are isofs and squashfs, but they're not as far along...)<br>
<br>
> > Another TODO item is packing up debootstrap and alpine root<br>
> filesystems to test<br>
> > under mkroot as more rigorous "TEST_HOST=1" runs. With the kernel<br>
> > config/version, mountpoint selection, and qemu board emulation parameters.<br>
> > Presumably running my init script instead of theirs to do the setup<br>
> and start<br>
> > the test, but using their $PATH of binaries (gnu/fsf and busybox,<br>
> respectively).<br>
> > But that's after I get the base mkroot testing well...<br>
> ><br>
> > > note that it's /etc/*group* that's weird, not *passwd*. uid 0 is<br>
> root, but<br>
> > group<br>
> > > 0 is wheel. (i think that's true of all bsds?)<br>
> <br>
> I suspect the easy solution is "skipnot grep -qw root /etc/group".<br>
> <br>
> Because it's not gonna make the same tarball otherwise, and this is testing data<br>
> fetched out of /etc/passwd. (We have --owner=NAME:ID tests: this isn't that.)<br>
> <br>
> wouldn't it make more sense to use one of the low-numbered groups that's the<br>
> same on both though, rather than skip?<br>
<br>
Grrr... trusting "root" to be there seemed as universal as you could get.<br>
Slowaris had "root". AIX had "root". I very vaguely recall Eunice on the VAX<br>
_thinking_ it had "root" although I was something like thirteen years old at the<br>
time so it's anybody's guess whether that's accurate. (I don't remember what my<br>
Uncle Alan's HP minicomputer in his den had when I was 7, I was just playing<br>
hangman on the teletype. Yes, actual printer.)<br>
<br>
Then again, I probably only remember looking at the USER root, who paid<br>
attention to groups after 1980? Having the user and group gratuitously different<br>
(but only for this ONE user, presumably daemon/daemon and bin/bin and so on<br>
match) is... a historical bug in BSD? As with having "eth0" not exist in /dev<br>
when everything ELSE is a file: turns out Bill Joy did not quite grok this unix<br>
thing...<br>
<br>
Anyway, I've got this todo item in the tar --transform bucket, lemme finish up<br>
with diff first. (And file off another couple rough edges on sh I've got<br>
in-progress...)<br>
<br>
> The second truncate makes the file NOT SPARSE.<br>
> <br>
> > the weird part for me was that it wasn't obvious _what_ the non-zero<br>
> number was<br>
> > going to be. <br>
> <br>
> The bug seems to be if you extend a sparse file the result is not sparse. I'm<br>
> guessing if you go:<br>
> <br>
> truncate -s 1k empty; truncate -s 2m empty; stat -c %b empty<br>
> <br>
> You'll get 4096. (Because 2048*512=1m and that's what we asked it to expand the<br>
> sparse file to.)<br>
> <br>
> <br>
> /tmp$ ~/toybox/toybox truncate -s 1k empty; ~/toybox/toybox truncate -s 2m<br>
> empty; ~/toybox/toybox stat -c %b empty<br>
> 4096<br>
<br>
Yup. Looks like a mac kernel bug to me. You can extend an empty file sparsely,<br>
and can extend a non-sparse file sparsely, but if you extend a sparse file it<br>
allocates all the zeroes.<br>
<br>
> > > iirc it's a bit simpler than that (if you don't have some company<br>
> policy that<br>
> > > says you can only install binaries from their servers) --- you just<br>
> run "make"<br>
> > > and it pops up a window saying "you want to install all that shit?" and<br>
> > you say<br>
> > > "it's a unix system; i know this", bish bash bosh, job done.<br>
<br>
Do you actually have to say the line from Jurassic park out loud? Or do you type<br>
it at the script?<br></blockquote><div><br></div><div>Apple's not that cool ... you just _think_ that while clicking "okay" :-) </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Rob<br>
</blockquote></div></div>