[Toybox] Android sitrep

[enh]

On Fri, Aug 30, 2019 at 6:39 PM Rob Landley <rob at landley.net> wrote:
>
> On 8/29/19 7:53 PM, enh wrote:
> > On Thu, Aug 29, 2019 at 5:30 PM Rob Landley <rob at landley.net> wrote:
> >>
> >> On 8/28/19 7:08 PM, enh via Toybox wrote:
> >>> i guess now's a good time for another sitrep, since i've been out sick
> >>> all week and so had time to poke at various things, and there's been
> >>> quite a lot going on:
> >>>
> >>> * after the genuine dd bug found by the continuous tests was fixed,
> >>> i've seen no more test failures there.
> >>
> >> Cool. I'll try to give you a heads up before I do a cleanup pass on anything
> >> android's building out of pending.
> >
> > don't worry, i'm not as crazy as you think i am ... pending is only
> > for on-device ("something is better than nothing"). there are a few
> > things in pending that could be used on the host, but my life is
> > enough of a plate-spinning exercise as it is without inviting trouble
> > :-)
> >
> > ("which things in pending?" specifically bc, dd, diff, expr, lsof, and tr.)
>
> The roadmap currently has:
>
>   chrt dd expr getfattr* lsof modprobe more setfattr* tar tr traceroute
>
> Which I should update because I got chrt, setfattr, and tar cleaned up. And I
> need to add bc and diff.
>
> The current Android.bp has:
>
>         "toys/pending/bc.c",
>         "toys/pending/dd.c",
>         "toys/pending/diff.c",
>         "toys/pending/expr.c",
>         "toys/pending/tr.c",
>
> And then later in the same file:
>
>                 "toys/pending/getfattr.c",
>                 "toys/pending/lsof.c",
>                 "toys/pending/modprobe.c",
>                 "toys/pending/more.c",
>                 "toys/pending/stty.c",
>                 "toys/pending/traceroute.c",
>
> You listed lsof out of the second list but not the others?

correct. lsof is (odd thought it may seem) used in the build --- but
only if something goes wrong; the resulting bug reporting includes
things like lsof and pstree output. so they're third-class citizens as
far as i'm concerned:

1. non-pending, used in successful build
2. pending, used in successful build
3. pending, not used normally

> >> The android directory's your baby, but could you explain a bit more what you
> >> were using them for before, and where you're using them now?
> >
> > i used to just have one .config so i'd have to jump through a few
> > hoops to be able to use that on the linux host too. now i have
> > separate configs, Android stuff only needs to build on Android, so
> > portability shmortability.
>
> If they're not needed during the AOSP build and only have to be present on the
> target, then sure.

if anyone starts using traceroute as part of the build, we're going to
have to have words with them :-)

(i assume the nsjail they're run in prevents that kind of silliness anyway.)

> The reason toybox can do ps is it has magic selinux attributes only created
> during the AOSP build, not available in the NDK build. I take it all the selinux
> attributes in the new filesystem are created during the filesystem packaging step?

yes.

> (I tried to dig into AOSP recently. The Makefile sourced build/make/core/main.mk
> which started with ANDROID_BUILD_SHELL and I went off to look up what that was
> and wound up reading about https://developer.android.com/ndk/guides/wrap-script
> instead and never got back... :)
>
> >> Yay. In theory I'm doing that here, in practice it's sometimes just every couple
> >> weeks so I can take a while to spot regressions. And I'm testing vanilla, not
> >> android kernel, and not your config...
> >
> > (i don't actually know _which_ kernel they're building. it could
> > actually be upstream, or it could be the "android common kernel", or
> > it could be -- and this is actually the least likely -- a device
> > kernel.)
>
> I love reproducible bug reports. Sending real world data through the commands is
> how you find real world problems, I'm all for it.

i can tell you already that one of the issues is going to be xargs -P
--- i almost sent you a patch last week that made -P a no-op (given
that the docs say "up to ... processes"); i don't know whether it
actually makes a difference to build times.

i'll pass on proper bug reports when i have them.

> >>>       * anecdotally i've heard that tar and xargs (independently) gave
> >>> them problems, but i've yet to receive the promised bug reports.
> >>
> >> I have a tar bug report pending through github. I think I'm caught up on your
> >> xargs patches?
> >
> > yeah, other than the ls and find patches i sent you today (the ls one
> > found because my attempt to sync with github toybox failed presubmit,
> > and the find one found as the next issue with my attempt to switch the
> > host over to building with toybox find), i think everything i've sent
> > you is in.
>
> I applied 4 patches this morning, hopefully that's caught up.

yeah, i let the machines churn again overnight and they're happy.
(i've refrained from merging the changes yet though, since the rest of
the world is actually at work this monday, even if the US isn't!)

> > (well, there's also the README.md thing that you should
> > either accept or close the bug and say "not going to happen" [or
> > explain what changes to my patch you'd like
>
>
> Yup, that's in the prerelease pile for this weekend.
>
> I applied it locally and converted it with the "markdown" command to html and
> pointed chrome at it with file:/// and got nearly imperceptible font shifts that
> are apparently supposed to mean something? (Is there a missing stylesheet it
> expects maybe?) Plus a lot of breakage like:
>
>   Toybox uses the make menuconfig; make; make install idiom same as the Linux
>   kernel. Usually you want something like:
>
>   make defconfig make make install
>
>   Or maybe:
>
> It looks like there's a missing <blockquote> tag in the output...?
>
> Presumably the response is "oh the markdown command is broken, github does it
> magically right so you can only see what your changes look like after you upload
> them to an external website"... which has kinda slowed my acceptance of this
> patch. :P

so much as i think markdown is the least worst option for markup ...
the biggest problem is that there is no One True Markdown. aiui, the
original one is so anemic that no-one uses it, and there are a bunch
of independent reimplementations with no conformance test suite.
github and gitiles are different, for example, which are the two i
dealt with regularly.

_but_ there is a common subset that you can rely on.

i suspect here you want to use ``` to blockquote.

(but i can tell you that what i sent you does work with github because
i edited it in the github bug tracker. you can just switch between
"write" and "preview" there. and github at least uses the same
markdown implementation everywhere :-) )

> Sigh. I'm probably going to have to push to a branch, view the readme in the
> branch on the website, delete the branch, and then push normally. Which makes it
> hard to think of this as anything other than Github By Microsoft's proprietary
> documentation format...
>
> (I also installed python-markdown but the output only differed by whitespace.
> The man page doesn't seem to have a --pretty option or anything?)
>
> > --- that's perfect work
> > for https://en.wikipedia.org/wiki/Homes_Under_the_Hammer ].)
>
> I watched https://www.hgtv.com/shows/holmes-on-homes instead. Sadly no longer on
> netflix. (Fade really likes "Paranormal home inspectors" where a psychic, a
> paranormal investigator, and a certified home inspector investigate ghost
> sightings. The first two do woo-woo while the inspector explains they have
> raccoons or something.
> https://observer.com/2014/10/the-best-worst-show-on-netflix-is-paranormal-home-inspectors/
> )

(Lord "educate, inform, and entertain, in that order" Reith would spin
in his grave if he knew about Homes Under the Hammer... hard to
believe that the BBC is encouraging people to buy houses at auction,
downplay the amount of risk and work involved, and sweep all the
incidental costs under the carpet.)

> >> I await this opening of this can of worms, but an probably going to try to cut a
> >> release first.
> >
> > no, this shouldn't affect you. linux is already switched over, the mac
> > binary passes the tests. all i need to do is clean up AOSP to not
> > assume the different mac date/stat behaviors. basically just me
> > deleting stuff from build files. but _finding_ the stuff to delete
> > will be the annoying part...
>
> "One of my most productive days was throwing away 1000 lines of code." - Ken
> Thompson
>
> (Which the internet seem unsure is a real quote, but I saw it in Ken's
> handwriting. Back when I was editing The Art of Unix Programming in 2003, the
> publisher sent a printout of the first 8 chapters to Ken Thompson for review,
> and he sent them back marked up in red pen. That was a note his wrote in the
> margin in response to something we'd said, and we liked it so much we put the
> quote from him in the text.)
>
> >>> * unrelated, i built a static toybox binary.
> >>
> >> Yay! (Against bionic?)
> >
> > correct. the current (R, not Q!) libc.a.
>
> I'm all for it.

the NDK guy is too, so even though i'm a bit unsure about the
usefulness without adbd fixes on old devices (which aren't possible)
and a mksh static binary (which i could also do), i'll get this done.
"something is better than nothing", after all.

> >>>   * the transitive dependencies are pretty gross thanks to the cgroup
> >>> stuff. i wonder if anyone uses that enough for it to be really worth
> >>> it, or whether we could drop it without anyone noticing?
> >>
> >> Huh, I did this in nsenter:
> >>
> >> #define unshare(flags) syscall(SYS_unshare, flags)
> >> #define setns(fd, nstype) syscall(SYS_setns, fd, nstype)
> >>
> >> Precisely to _reduce_ dependencies. Do you mean the CLONE flags being defined,
> >> or...?
> >
> > no, the Android cgroup stuff. grep ps.c for ANDROID. basically so ps
> > can say "bg" or "fg" or whatever rather than an int.
>
> In all caps there's just the one?
>
>   else if (CFG_TOYBOX_ON_ANDROID)
>     sprintf(not_o = toybuf+128,
>             "USER,%%sPPID,VSIZE,RSS,WCHAN:10,ADDR:10,S,%s",
>             FLAG(T) ? "CMD" : "NAME");
>
> The other case insensitive hits are about scheduling policy and a todo about
> field size?
>
> bg and fg... the shell commands?

no, the background and foreground cgroups used for scheduling policy.
(the first of the case-insensitive matches you found.)

> I've missed a curve here.
>
> >>>   * ran tests (with some difficulty) on jellybean, and had about 100
> >>> failures out of about 1200 tests. (`adb shell` didn't return exit
> >>> status that far back, so this is grep for PASS and FAIL rather than
> >>> the usual output from my little test runner.)
> >>
> >> Statically linked against a newer bionic, but running on the older kernel?
> >
> > correct.
>
> Seems pretty reasonable then. Do you have a dump of what the failures are so we
> can categorize them?

not yet, i was doing it in parallel with a bunch of other stuff. i'll
have a proper run though now everyone seems in favor of doing this.

> >> You sent a link to https://developer.android.com/about/dashboards which says
> >> Jellybean is 3.2% of machines checking into the play store or some such. Even
> >> adding in the 0.6% from the previous 2 releases, that means 96% of all android
> >> devices are running kitkat or newer.
> >
> > aye, but our self-imposed albatross is that we wait until the share is < 1%.
>
> Works for me. (And technically the 7 year horizon agrees with you. :)
>
> >> While the 7 year horizon is a bit over Kit Kat's introduction 6 years and 1
> >> month ago, historical static binary thing is a new feature. I'm ok anchoring it
> >> to kitkat if you are. That's still going back 2 releases before toybox was
> >> _introduced_.
> >
> > ...which is exactly why those devices need a static toybox the most :-)
> >
> > i wish i had a good idea of "oldest release anyone's doing any serious
> > testing on", but i don't. (and like i say, the shell and adb are awful
> > that far back too. i don't think adb was fixed until N :-( )
>
> I'm in the process of writing a proper shell you can |& with, which can handle
> its own job control and such. I have:
>
>   $ cat boing |& sed s/boing/walrus/
>   cat: walrus: No such file or directory
>
> In my test pile. Alas, it is by _no_ means there yet, but I am at least working
> on it. :)

the shell's actually the easy part: it's easier to package that up
than a static toybox (because fewer dependencies, and especially not
that awkward libcutils one). i pity anyone who has to do any automated
testing on old releases. (this is why the world is full of weird stuff
like parsing ls(1) output to test whether files exist... <movie
trailer voice>i a world where you can't rely on exit status...)

> A thing I've had on my todo list for a while is a pty wrapper that can create a
> pty master and call a child process with fake tty info, and turn the tty
> requests into some sort of annotation on the output. There's some significant
> design work there, but without it I dunno how you test "top" and such.
>
> Closest I've found out there is https://github.com/martinpitt/umockdev which is
> mostly addressing stuff I'm trying to throw QEMU at instead. But a closer look
> is on the todo list...
>
> >>>   * a bunch of the failures are because the current tz code in libc.a
> >>> doesn't support the tz data format back in jellybean. i _think_ kitkat
> >>> was the transition, so i'd like to rerun the tests when i've had
> >>> chance to plug a kitkat device in at my desk.
> >>>
> >>>   * regardless, it seems like it might be plausibly useful to make a
> >>> static toybox available on ci.android.com. don't know whether you'd
> >>> want to link to that (which was why i pinged you on that bug about
> >>> switching to README.md like all the cool kids).
> >>
> >> I'm happy to do that, and from the website's nav bar. And yes, I need to catch
> >> up on patches this weekend, that's one in the stack for this release.
> >
> > it's certainly easier than folks trying to build toybox themselves...
>
> I need to get the FAQ updated with build instructions for various targets. (It's
> in the README, but more docs doesn't hurt...)

toybox is an example in the NDK docs, but i'm not sure how many people
actually build it themselves rather than just suffer whatever happened
to be on whatever device they have.

> Rob