<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Jun 25, 2022 at 3:45 PM Rob Landley <<a href="mailto:rob@landley.net">rob@landley.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 6/24/22 19:35, enh wrote:<br>
> On Thu, Jun 23, 2022 at 11:48 PM Rob Landley <<a href="mailto:rob@landley.net" target="_blank">rob@landley.net</a><br>
> <mailto:<a href="mailto:rob@landley.net" target="_blank">rob@landley.net</a>>> wrote:<br>
> <br>
> On 6/22/22 20:02, enh wrote:<br>
> > On Wed, Jun 22, 2022 at 1:52 PM Rob Landley <<a href="mailto:rob@landley.net" target="_blank">rob@landley.net</a><br>
> <mailto:<a href="mailto:rob@landley.net" target="_blank">rob@landley.net</a>><br>
> > <mailto:<a href="mailto:rob@landley.net" target="_blank">rob@landley.net</a> <mailto:<a href="mailto:rob@landley.net" target="_blank">rob@landley.net</a>>>> wrote:<br>
> <br>
> The problem with the mac tar test is even though it's easy enough to find what<br>
> /etc/passwd calls UID 0:<br>
> <br>
> ROOT="$(sed -n '/[^:]*:[^:]*:0:/s/:.*//p' /etc/passwd)"<br>
> <br>
> That doesn't change the fact it'll be putting a different string into the<br>
> tarball, with different sha1sums. Um. (I was using "root" as the one known<br>
> constant account that didn't vary across distros. Possibly I need a way to tell<br>
> it to use an alternate /etc/passwd file to lookup usernames. This is why I've<br>
> been poking at mkroot, but making that work on a mac is just... ow.)<br>
<br>
FYI, I committed your patch shortly after sending that message.<br></blockquote><div><br></div><div>thanks. interestingly, i realized that i think we also wouldn't get a red cross in the github ui if we broke the _linux_ tests? it's only a build failure that counts? not obvious to me from the .yaml syntax why that is/what we could do about it.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I can't immediately think of a better short-term fix, with the possible<br>
exception of tagging tests as "known to fail on macos because macos is buggy".<br>
(And "we extended a zero length file three times with truncate() and along the<br>
way it allocated a megabyte of storage to store LITERALLY NO DATA" sounds like a<br>
bug to me. I am neither interested in fixing nor reporting MacOS bugs because<br>
they're 100% proprietary with 0% open source input, and they ain't paying me to<br>
make them richer thanks. For the same reason, I don't want to put a lot more<br>
cycles into _thinking_ about macos either.)<br>
<br>
The mkroot stuff is all about "I can mount ext2 or tmpfs to run this test on and<br>
have exactly known behavior". I understand "somebody ran the test on xfs and it<br>
behaved differently than any other filesystem so far", but I think this is a bug<br>
in the VFS layer in a test environment I haven't got. When porting tests into<br>
mkroot, I'd presumably do some annotation for "this test runs in the<br>
known/mkroot environment" anyway, and logically I'd tag the ones that have known<br>
problems outside that environment, whatever those problems may be...<br></blockquote><div><br></div><div>or we could have a more specific fs-specific "skipnot", since "what fs is this?" seems to be one of the most common problems.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Another TODO item is packing up debootstrap and alpine root filesystems to test<br>
under mkroot as more rigorous "TEST_HOST=1" runs. With the kernel<br>
config/version, mountpoint selection, and qemu board emulation parameters.<br>
Presumably running my init script instead of theirs to do the setup and start<br>
the test, but using their $PATH of binaries (gnu/fsf and busybox, respectively).<br>
But that's after I get the base mkroot testing well...<br>
<br>
> note that it's /etc/*group* that's weird, not *passwd*. uid 0 is root, but group<br>
> 0 is wheel. (i think that's true of all bsds?)<br>
> <br>
> it actually looks like gid 0 is one of the few low gids that differs?<br>
<br>
Alas I believe this is one of the holes in posix, because Windows NT and System<br>
360 did not have user and group names but they REALLY wanted that FIPS 151-2 money.<br>
<br>
They specify ways to change uid and gid of resources, but don't ever say what<br>
any of those IDs _are_. (Unless I'm missing it? They handwave "appropriate<br>
privileges" and such instead of saying uid 0 or root...)<br>
<br>
> here's macos' first few gids:<br>
> <br>
> nobody:*:-2:<br>
> nogroup:*:-1:<br>
> wheel:*:0:root<br>
> daemon:*:1:root<br>
> kmem:*:2:root<br>
> sys:*:3:root<br>
> tty:*:4:root<br>
> <br>
> most of those are the same on linux.<br>
<br>
I'm not spotting any negative gids in /etc/group on devuan. And I think that<br>
violates posix?<br>
<br>
<a href="https://pubs.opengroup.org/onlinepubs/9699919799/functions/chown.html" rel="noreferrer" target="_blank">https://pubs.opengroup.org/onlinepubs/9699919799/functions/chown.html</a><br>
<br>
chown(-1) means "don't change". So you can't set it to -1 through the posix<br>
specified API.<br></blockquote><div><br></div><div>they're just taking advantage of a scanf("%u") somewhere else. the interesting part is that this means those _aren't_ actually the 64Ki and 64Ki-1 i was expecting:</div><div><br></div><div>~$ id nobody<br>uid=4294967294(nobody) gid=4294967294(nobody) groups=4294967294(nobody),12(everyone),61(localaccounts),100(_lpoperator)<br></div><div><br></div><div>but the small positive ones look okay?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> > On 6/20/22 21:32, enh via Toybox wrote:<br>
> > > Specifically we were ending up with 2048 blocks allocated, and that --<br>
> > > not the stat(1) behavior -- was the reason why this test was failing<br>
> > > on macOS.<br>
> ><br>
> > Yay, I got thunderbird sending email through gmail again! (The version<br>
> upgrade<br>
> > changed the authentication type in the smtp server settings, but I<br>
> could still<br>
> > set it BACK...)<br>
> ><br>
> > I wanted a partially sparse file there, which truncate was preserving the<br>
> > contents of.<br>
> ><br>
> > i must be missing something ... why is it _partially_ sparse?<br>
> <br>
> Because truncate should be able to extend an existing file sparsely without<br>
> damaging the existing contents, but I see that the existing test wasn't doing<br>
> what I thought it was doing...<br>
> <br>
> So hang on, if we made a file completely sparse in 3 segments, how many blocks<br>
> does MacOS think it's allocated? Can I dig this out of the macos test log on<br>
> github?<br>
> <br>
> <a href="https://github.com/landley/toybox/runs/7008521598?check_suite_focus=true" rel="noreferrer" target="_blank">https://github.com/landley/toybox/runs/7008521598?check_suite_focus=true</a><br>
> <<a href="https://github.com/landley/toybox/runs/7008521598?check_suite_focus=true" rel="noreferrer" target="_blank">https://github.com/landley/toybox/runs/7008521598?check_suite_focus=true</a>><br>
> <br>
> scripts/test.sh<br>
> scripts/runtest.sh: line 223: syntax error near unexpected token `;'<br>
> scripts/runtest.sh: line 223: ` R) LEN=0; B=1; ;&'<br>
> <br>
> It _starts_ with a syntax error, and THEN proceeds to run the tests. I don't<br>
> understand what it's doing here?<br>
> <br>
> (that's similar to what you see on Android with mksh at the moment --- there's<br>
> some error but apparently it's harmless because the tests run anyway.)<br>
<br>
Another test case to file under "syntax error abort granularity":<br>
<br>
$ cat one<br>
#!/bin/bash<br>
source two<br>
echo hello<br>
$ cat two<br>
syntax error)<br>
$ bash one<br>
two: line 1: syntax error near unexpected token `)'<br>
two: line 1: `syntax error)'<br>
hello<br>
<br>
You lose txpect, mkchroot, and dochroot. None of which those particular tests<br>
are using...<br>
<br>
> <br>
> Scroll, scroll, scroll through the test results. VERBOSE=allnopass is probably<br>
> your friend here...<br>
> <br>
> No it does not. And I don't have a mac to try this on. Hmmm...<br>
> <br>
> happy to run commands if you send me them.<br>
<br>
$ rm -f empty; for i in 1k 1m 1g; do truncate -s $i empty; stat -c %b empty;<br>
done; ls -l empty<br>
0<br>
0<br>
0<br>
-rw-r--r-- 1 landley landley 1073741824 Jun 25 17:15 empty<br>
<br>
I'm guessing it's not gonna say 0.<br></blockquote><div><br></div><div>~/toybox$ rm -f empty; for i in 1k 1m 1g; do ./toybox truncate -s $i empty; ./toybox stat -c %b empty; done; ls -l empty<br>0<br>2048<br>2048<br>-rw-r--r-- 1 enh primarygroup 1073741824 Jun 27 12:38 empty<br>~/toybox$</div><div><br></div><div>the weird part for me was that it wasn't obvious _what_ the non-zero number was going to be. <br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> > The problem is, you're not using bash or toysh for these tests, you're<br>
> using a<br>
> > shell of unknown capabilities. Can the macos and android shells do that?<br>
> ><br>
> > macos _is_ bash, it's just 3.2.57 from 2007, for exactly the reason you'd<br>
> assume.<br>
> <br>
> They swapped to zsh three years ago:<br>
> <br>
> <a href="https://www.theverge.com/2019/6/4/18651872/apple-macos-catalina-zsh-bash-shell-replacement-features" rel="noreferrer" target="_blank">https://www.theverge.com/2019/6/4/18651872/apple-macos-catalina-zsh-bash-shell-replacement-features</a><br>
> <br>
> <br>
> oh, yeah, good point. my reaction to that was similar to your reaction to dash.<br>
> _i'll_ be using bash on macos until they remove it.<br>
<br>
All my stuff says #!/bin/bash at the top, but I dunno how github is running what...<br></blockquote><div><br></div><div>just `VERBOSE=all make tests`<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> That said I THINK all this $((math)) stuff I've been proposing is posix? So it<br>
> should work...?<br>
> <br>
> yeah, certainly your <br>
> <br>
> x=1+3; echo $((x))<br>
> <br>
> example works the same on both zsh and old bash (printing 4) on this mac.<br>
<br>
And on current toysh. :)<br>
<br>
> > android is mksh, which is available as a debian package; i often do "android"<br>
> > shell testing that way for convenience :-)<br>
> ><br>
> > if you send me a patch i can test it for you ... but feel free to just commit<br>
> > "the right thing" and we can work from there!<br>
> <br>
> I'm no longer convinced I know what the right thing is here. Possibly the<br>
> "delete and recreate" workaround is best, but I have no idea WHY it works...<br>
> <br>
> I'm tempted to borrow my wife's mac for a bit, but I have no idea how to set up<br>
> a development environment on a mac. The first google hit is<br>
> <a href="https://sourabhbajaj.com/mac-setup/Xcode/" rel="noreferrer" target="_blank">https://sourabhbajaj.com/mac-setup/Xcode/</a> which looks... more elaborate than I<br>
> want to do on a borrowed machine.<br>
> <br>
> iirc it's a bit simpler than that (if you don't have some company policy that<br>
> says you can only install binaries from their servers) --- you just run "make"<br>
> and it pops up a window saying "you want to install all that shit?" and you say<br>
> "it's a unix system; i know this", bish bash bosh, job done.<br>
<br>
Except I can't easily _undo_ it afterwards and don't want to eat I dunno how<br>
many gigs on my wife's machine with Apple's soldered-in ssd.<br></blockquote><div><br></div><div>yeah, it's not small.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
*shrug* I can ask...<br>
<br>
> or, like i say, commit something that works on linux and i'll be your test monkey.<br>
<br>
I committed the patch you sent me. It has a big enough comment I can clean it up<br>
on a future pass through...<br>
<br>
> Rob<br>
<br>
Rob<br>
</blockquote></div></div>