[Toybox] Android sitrep

[Rob Landley]

On 8/28/19 7:08 PM, enh via Toybox wrote:
> i guess now's a good time for another sitrep, since i've been out sick
> all week and so had time to poke at various things, and there's been
> quite a lot going on:
> 
> * after the genuine dd bug found by the continuous tests was fixed,
> i've seen no more test failures there.

Cool. I'll try to give you a heads up before I do a cleanup pass on anything
android's building out of pending.

> * md5sum and sha*sum were motivating cases to actually properly
> support macOS builds.
> 
>   * i now, for my sins, have three different .config files, three
> different generated/ directories, and a short script using NOBUILD to
> regenerate the generated/ files.

Android target, linux 86 host, macos host?

(Does the windows build happen under that eniw "linux services for windows" thing?)

>     * the good news about that is that i no longer have any need to
> build toys/android/ stuff on the host. would you like me to clean up
> some of the compatibility cruft that (iirc) is only there to support
> that? if it's not useful to me, i'm assuming it's not useful to
> anyone?

The android directory's your baby, but could you explain a bit more what you
were using them for before, and where you're using them now?

>     * the bad news that turned into good news was that this
> accidentally regressed someone's kernel builds (because AOSP doesn't
> need nproc, and i cleaned up the host configs to only include tools
> actually used, not everything that goes on the device). i added nproc
> back to the linux host toybox, and the person who was affected added a
> kernel build to the presubmit check for a toybox prebuilt update. so
> if we break kernel builds in some way we won't see it when i sync with
> github, but we will see it when the build folks try to build a new
> prebuilt. which is still fairly frequently.

Yay. In theory I'm doing that here, in practice it's sometimes just every couple
weeks so I can take a while to spot regressions. And I'm testing vanilla, not
android kernel, and not your config...

>       * anecdotally i've heard that tar and xargs (independently) gave
> them problems, but i've yet to receive the promised bug reports.

I have a tar bug report pending through github. I think I'm caught up on your
xargs patches?

>       * it's still not 100% clear to me whether they consider using
> toybox for these builds to be deliberate or accidental. it happened by
> accident (in search of a hermetic build), but they've kept it for now
> so ... deliberate?

Yay?

I'm still trying to go beyond hermetic builds to self hosting. Baby steps.

>   * there's a macOS toybox prebuilt now too, updated whenever the linux one is.

Ok.

>   * i _added_ md5sum and sha*sum which the mac doesn't normally have
> (the slightly different md5 and shasum instead).
> 
>   * i switched over everything used in the AOSP build that's currently
> toybox on linux to being toybox on the mac too (except date and stat).
> that's only been in a day, but it hasn't had to be reverted yet.
> 
>   * date and stat need a bit of tree cleanup because there are places
> that _expect_ differences in behavior.

I await this opening of this can of worms, but an probably going to try to cut a
release first.

> * unrelated, i built a static toybox binary.

Yay! (Against bionic?)

>   * the transitive dependencies are pretty gross thanks to the cgroup
> stuff. i wonder if anyone uses that enough for it to be really worth
> it, or whether we could drop it without anyone noticing?

Huh, I did this in nsenter:

#define unshare(flags) syscall(SYS_unshare, flags)
#define setns(fd, nstype) syscall(SYS_setns, fd, nstype)

Precisely to _reduce_ dependencies. Do you mean the CLONE flags being defined,
or...?

>   * the chips in those devices are old enough that you need a generic
> build to not die immediately with SIGILL. (i didn't have any of my
> usual tools to work out _what_ the invalid instruction is.)

I had this problem with my netbook last year:

http://lists.landley.net/pipermail/toybox-landley.net/2018-March/009437.html
https://landley.net/notes-2018.html#12-03-2018

Said netbook's cmos battery finally gave out shortly thereafter and I moved to a
new machine, so it was kinda mooted. If I recall the problem was even if you
sent an -mcpu flag to llvm, the startup code it was sucking in (crt1.o or some
such) had the illegal instruction in it.

That said, the startup code being more generic wouldn't be a _bad_ thing...

>   * ran tests (with some difficulty) on jellybean, and had about 100
> failures out of about 1200 tests. (`adb shell` didn't return exit
> status that far back, so this is grep for PASS and FAIL rather than
> the usual output from my little test runner.)

Statically linked against a newer bionic, but running on the older kernel?

You sent a link to https://developer.android.com/about/dashboards which says
Jellybean is 3.2% of machines checking into the play store or some such. Even
adding in the 0.6% from the previous 2 releases, that means 96% of all android
devices are running kitkat or newer.

While the 7 year horizon is a bit over Kit Kat's introduction 6 years and 1
month ago, historical static binary thing is a new feature. I'm ok anchoring it
to kitkat if you are. That's still going back 2 releases before toybox was
_introduced_.

>   * a bunch of the failures are because the current tz code in libc.a
> doesn't support the tz data format back in jellybean. i _think_ kitkat
> was the transition, so i'd like to rerun the tests when i've had
> chance to plug a kitkat device in at my desk.
> 
>   * regardless, it seems like it might be plausibly useful to make a
> static toybox available on ci.android.com. don't know whether you'd
> want to link to that (which was why i pinged you on that bug about
> switching to README.md like all the cool kids).

I'm happy to do that, and from the website's nav bar. And yes, I need to catch
up on patches this weekend, that's one in the stack for this release.

I really wanted to get toysh running the darn mkroot init script before cutting
a release (it's 30 lines!
https://github.com/landley/toybox/blob/master/scripts/mkroot.sh#L68 ) but I'm
unlikely to hit that goal this weekend and it looks like I fly to Tokyo for work
sometime next week. Gotta get a release out before then...

Rob