[Toybox] [PATCH] Fix truncate.test for macOS.
enh
enh at google.com
Mon Jun 27 12:40:55 PDT 2022
On Sat, Jun 25, 2022 at 3:45 PM Rob Landley <rob at landley.net> wrote:
> On 6/24/22 19:35, enh wrote:
> > On Thu, Jun 23, 2022 at 11:48 PM Rob Landley <rob at landley.net
> > <mailto:rob at landley.net>> wrote:
> >
> > On 6/22/22 20:02, enh wrote:
> > > On Wed, Jun 22, 2022 at 1:52 PM Rob Landley <rob at landley.net
> > <mailto:rob at landley.net>
> > > <mailto:rob at landley.net <mailto:rob at landley.net>>> wrote:
> >
> > The problem with the mac tar test is even though it's easy enough to
> find what
> > /etc/passwd calls UID 0:
> >
> > ROOT="$(sed -n '/[^:]*:[^:]*:0:/s/:.*//p' /etc/passwd)"
> >
> > That doesn't change the fact it'll be putting a different string
> into the
> > tarball, with different sha1sums. Um. (I was using "root" as the one
> known
> > constant account that didn't vary across distros. Possibly I need a
> way to tell
> > it to use an alternate /etc/passwd file to lookup usernames. This is
> why I've
> > been poking at mkroot, but making that work on a mac is just... ow.)
>
> FYI, I committed your patch shortly after sending that message.
>
thanks. interestingly, i realized that i think we also wouldn't get a red
cross in the github ui if we broke the _linux_ tests? it's only a build
failure that counts? not obvious to me from the .yaml syntax why that
is/what we could do about it.
> I can't immediately think of a better short-term fix, with the possible
> exception of tagging tests as "known to fail on macos because macos is
> buggy".
> (And "we extended a zero length file three times with truncate() and along
> the
> way it allocated a megabyte of storage to store LITERALLY NO DATA" sounds
> like a
> bug to me. I am neither interested in fixing nor reporting MacOS bugs
> because
> they're 100% proprietary with 0% open source input, and they ain't paying
> me to
> make them richer thanks. For the same reason, I don't want to put a lot
> more
> cycles into _thinking_ about macos either.)
>
> The mkroot stuff is all about "I can mount ext2 or tmpfs to run this test
> on and
> have exactly known behavior". I understand "somebody ran the test on xfs
> and it
> behaved differently than any other filesystem so far", but I think this is
> a bug
> in the VFS layer in a test environment I haven't got. When porting tests
> into
> mkroot, I'd presumably do some annotation for "this test runs in the
> known/mkroot environment" anyway, and logically I'd tag the ones that have
> known
> problems outside that environment, whatever those problems may be...
>
or we could have a more specific fs-specific "skipnot", since "what fs is
this?" seems to be one of the most common problems.
> Another TODO item is packing up debootstrap and alpine root filesystems to
> test
> under mkroot as more rigorous "TEST_HOST=1" runs. With the kernel
> config/version, mountpoint selection, and qemu board emulation parameters.
> Presumably running my init script instead of theirs to do the setup and
> start
> the test, but using their $PATH of binaries (gnu/fsf and busybox,
> respectively).
> But that's after I get the base mkroot testing well...
>
> > note that it's /etc/*group* that's weird, not *passwd*. uid 0 is root,
> but group
> > 0 is wheel. (i think that's true of all bsds?)
> >
> > it actually looks like gid 0 is one of the few low gids that differs?
>
> Alas I believe this is one of the holes in posix, because Windows NT and
> System
> 360 did not have user and group names but they REALLY wanted that FIPS
> 151-2 money.
>
> They specify ways to change uid and gid of resources, but don't ever say
> what
> any of those IDs _are_. (Unless I'm missing it? They handwave "appropriate
> privileges" and such instead of saying uid 0 or root...)
>
> > here's macos' first few gids:
> >
> > nobody:*:-2:
> > nogroup:*:-1:
> > wheel:*:0:root
> > daemon:*:1:root
> > kmem:*:2:root
> > sys:*:3:root
> > tty:*:4:root
> >
> > most of those are the same on linux.
>
> I'm not spotting any negative gids in /etc/group on devuan. And I think
> that
> violates posix?
>
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/chown.html
>
> chown(-1) means "don't change". So you can't set it to -1 through the posix
> specified API.
>
they're just taking advantage of a scanf("%u") somewhere else. the
interesting part is that this means those _aren't_ actually the 64Ki and
64Ki-1 i was expecting:
~$ id nobody
uid=4294967294(nobody) gid=4294967294(nobody)
groups=4294967294(nobody),12(everyone),61(localaccounts),100(_lpoperator)
but the small positive ones look okay?
> > > On 6/20/22 21:32, enh via Toybox wrote:
> > > > Specifically we were ending up with 2048 blocks allocated,
> and that --
> > > > not the stat(1) behavior -- was the reason why this test was
> failing
> > > > on macOS.
> > >
> > > Yay, I got thunderbird sending email through gmail again! (The
> version
> > upgrade
> > > changed the authentication type in the smtp server settings,
> but I
> > could still
> > > set it BACK...)
> > >
> > > I wanted a partially sparse file there, which truncate was
> preserving the
> > > contents of.
> > >
> > > i must be missing something ... why is it _partially_ sparse?
> >
> > Because truncate should be able to extend an existing file sparsely
> without
> > damaging the existing contents, but I see that the existing test
> wasn't doing
> > what I thought it was doing...
> >
> > So hang on, if we made a file completely sparse in 3 segments, how
> many blocks
> > does MacOS think it's allocated? Can I dig this out of the macos
> test log on
> > github?
> >
> >
> https://github.com/landley/toybox/runs/7008521598?check_suite_focus=true
> > <
> https://github.com/landley/toybox/runs/7008521598?check_suite_focus=true>
> >
> > scripts/test.sh
> > scripts/runtest.sh: line 223: syntax error near unexpected token `;'
> > scripts/runtest.sh: line 223: ` R) LEN=0; B=1; ;&'
> >
> > It _starts_ with a syntax error, and THEN proceeds to run the tests.
> I don't
> > understand what it's doing here?
> >
> > (that's similar to what you see on Android with mksh at the moment ---
> there's
> > some error but apparently it's harmless because the tests run anyway.)
>
> Another test case to file under "syntax error abort granularity":
>
> $ cat one
> #!/bin/bash
> source two
> echo hello
> $ cat two
> syntax error)
> $ bash one
> two: line 1: syntax error near unexpected token `)'
> two: line 1: `syntax error)'
> hello
>
> You lose txpect, mkchroot, and dochroot. None of which those particular
> tests
> are using...
>
> >
> > Scroll, scroll, scroll through the test results. VERBOSE=allnopass
> is probably
> > your friend here...
> >
> > No it does not. And I don't have a mac to try this on. Hmmm...
> >
> > happy to run commands if you send me them.
>
> $ rm -f empty; for i in 1k 1m 1g; do truncate -s $i empty; stat -c %b
> empty;
> done; ls -l empty
> 0
> 0
> 0
> -rw-r--r-- 1 landley landley 1073741824 Jun 25 17:15 empty
>
> I'm guessing it's not gonna say 0.
>
~/toybox$ rm -f empty; for i in 1k 1m 1g; do ./toybox truncate -s $i empty;
./toybox stat -c %b empty; done; ls -l empty
0
2048
2048
-rw-r--r-- 1 enh primarygroup 1073741824 Jun 27 12:38 empty
~/toybox$
the weird part for me was that it wasn't obvious _what_ the non-zero number
was going to be.
> > > The problem is, you're not using bash or toysh for these
> tests, you're
> > using a
> > > shell of unknown capabilities. Can the macos and android
> shells do that?
> > >
> > > macos _is_ bash, it's just 3.2.57 from 2007, for exactly the
> reason you'd
> > assume.
> >
> > They swapped to zsh three years ago:
> >
> >
> https://www.theverge.com/2019/6/4/18651872/apple-macos-catalina-zsh-bash-shell-replacement-features
> >
> >
> > oh, yeah, good point. my reaction to that was similar to your reaction
> to dash.
> > _i'll_ be using bash on macos until they remove it.
>
> All my stuff says #!/bin/bash at the top, but I dunno how github is
> running what...
>
just `VERBOSE=all make tests`
> > That said I THINK all this $((math)) stuff I've been proposing is
> posix? So it
> > should work...?
> >
> > yeah, certainly your
> >
> > x=1+3; echo $((x))
> >
> > example works the same on both zsh and old bash (printing 4) on this mac.
>
> And on current toysh. :)
>
> > > android is mksh, which is available as a debian package; i often
> do "android"
> > > shell testing that way for convenience :-)
> > >
> > > if you send me a patch i can test it for you ... but feel free to
> just commit
> > > "the right thing" and we can work from there!
> >
> > I'm no longer convinced I know what the right thing is here.
> Possibly the
> > "delete and recreate" workaround is best, but I have no idea WHY it
> works...
> >
> > I'm tempted to borrow my wife's mac for a bit, but I have no idea
> how to set up
> > a development environment on a mac. The first google hit is
> > https://sourabhbajaj.com/mac-setup/Xcode/ which looks... more
> elaborate than I
> > want to do on a borrowed machine.
> >
> > iirc it's a bit simpler than that (if you don't have some company policy
> that
> > says you can only install binaries from their servers) --- you just run
> "make"
> > and it pops up a window saying "you want to install all that shit?" and
> you say
> > "it's a unix system; i know this", bish bash bosh, job done.
>
> Except I can't easily _undo_ it afterwards and don't want to eat I dunno
> how
> many gigs on my wife's machine with Apple's soldered-in ssd.
>
yeah, it's not small.
> *shrug* I can ask...
>
> > or, like i say, commit something that works on linux and i'll be your
> test monkey.
>
> I committed the patch you sent me. It has a big enough comment I can clean
> it up
> on a future pass through...
>
> > Rob
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20220627/f5d0252b/attachment-0001.htm>
More information about the Toybox
mailing list