[Toybox] Android sitrep

Sun Oct 6 11:39:46 PDT 2019

I have soooo many open but unfinished email reply windows. This one's a month
old! (You can tell how much time I've had to "clean" by how many half-finished
open windows my laptop's got...)

On 9/1/19 12:57 PM, enh wrote:
>> You listed lsof out of the second list but not the others?
> 
> correct. lsof is (odd thought it may seem) used in the build --- but
> only if something goes wrong; the resulting bug reporting includes
> things like lsof and pstree output. so they're third-class citizens as
> far as i'm concerned:
> 
> 1. non-pending, used in successful build
> 2. pending, used in successful build
> 3. pending, not used normally

Ok.

>>>> The android directory's your baby, but could you explain a bit more what you
>>>> were using them for before, and where you're using them now?
>>>
>>> i used to just have one .config so i'd have to jump through a few
>>> hoops to be able to use that on the linux host too. now i have
>>> separate configs, Android stuff only needs to build on Android, so
>>> portability shmortability.
>>
>> If they're not needed during the AOSP build and only have to be present on the
>> target, then sure.
> 
> if anyone starts using traceroute as part of the build, we're going to
> have to have words with them :-)
> 
> (i assume the nsjail they're run in prevents that kind of silliness anyway.)
> 
>> The reason toybox can do ps is it has magic selinux attributes only created
>> during the AOSP build, not available in the NDK build. I take it all the selinux
>> attributes in the new filesystem are created during the filesystem packaging step?
> 
> yes.

Where is this packaging done in AOSP? (I can dig my way to it eventually, but
you'd probably know off the top of your head. :)

>> (I tried to dig into AOSP recently. The Makefile sourced build/make/core/main.mk
>> which started with ANDROID_BUILD_SHELL and I went off to look up what that was
>> and wound up reading about https://developer.android.com/ndk/guides/wrap-script
>> instead and never got back... :)
>>
>>>> Yay. In theory I'm doing that here, in practice it's sometimes just every couple
>>>> weeks so I can take a while to spot regressions. And I'm testing vanilla, not
>>>> android kernel, and not your config...
>>>
>>> (i don't actually know _which_ kernel they're building. it could
>>> actually be upstream, or it could be the "android common kernel", or
>>> it could be -- and this is actually the least likely -- a device
>>> kernel.)
>>
>> I love reproducible bug reports. Sending real world data through the commands is
>> how you find real world problems, I'm all for it.
> 
> i can tell you already that one of the issues is going to be xargs -P
> --- i almost sent you a patch last week that made -P a no-op (given
> that the docs say "up to ... processes"); i don't know whether it
> actually makes a difference to build times.

  While  xargs  is
  running,  you  can send its process a SIGUSR1 signal to increase
  the number of commands to run simultaneously, or  a  SIGUSR2  to
  decrease  the number.

Seriously?

Sigh, I'm ok with managing parallelism, you may notice that I'm doing so from a
bash script in scripts/make.sh:

  https://github.com/landley/toybox/blob/master/scripts/make.sh#L314

Ooh, when did wait -n show up? I really _wanted_ that when I was writing this,
and it wasn't available. Hmmm... looks like bash 4.3. Released Feb 2014. Hmmm.
Not quite 7 years (5.5 at this point). But once I can build toybox under toysh I
might make an exception. (Assuming it doesn't take that long to complete toysh.
Grrr.)

Anyway, I might whip up an xargs -P if it's quick and I get a gap, but this
SIGUSR increment/decrement thing is kind of _conceptually_ silly? (What... why...?)

>> Presumably the response is "oh the markdown command is broken, github does it
>> magically right so you can only see what your changes look like after you upload
>> them to an external website"... which has kinda slowed my acceptance of this
>> patch. :P
> 
> so much as i think markdown is the least worst option for markup ...
> the biggest problem is that there is no One True Markdown. aiui, the
> original one is so anemic that no-one uses it, and there are a bunch
> of independent reimplementations with no conformance test suite.
> github and gitiles are different, for example, which are the two i
> dealt with regularly.
> 
> _but_ there is a common subset that you can rely on.

How do I test what the output looks like locally?

[I got pointed at a thing since and have an open terminal window for that...]

> i suspect here you want to use ``` to blockquote.
> 
> (but i can tell you that what i sent you does work with github because
> i edited it in the github bug tracker. you can just switch between
> "write" and "preview" there. and github at least uses the same
> markdown implementation everywhere :-) )

Well that's something. Hmmm...

>>>>> * unrelated, i built a static toybox binary.
>>>>
>>>> Yay! (Against bionic?)
>>>
>>> correct. the current (R, not Q!) libc.a.
>>
>> I'm all for it.
> 
> the NDK guy is too, so even though i'm a bit unsure about the
> usefulness without adbd fixes on old devices (which aren't possible)
> and a mksh static binary (which i could also do), i'll get this done.
> "something is better than nothing", after all.

Yay!

>>>>>   * the transitive dependencies are pretty gross thanks to the cgroup
>>>>> stuff. i wonder if anyone uses that enough for it to be really worth
>>>>> it, or whether we could drop it without anyone noticing?
>>>>
>>>> Huh, I did this in nsenter:
>>>>
>>>> #define unshare(flags) syscall(SYS_unshare, flags)
>>>> #define setns(fd, nstype) syscall(SYS_setns, fd, nstype)
>>>>
>>>> Precisely to _reduce_ dependencies. Do you mean the CLONE flags being defined,
>>>> or...?
>>>
>>> no, the Android cgroup stuff. grep ps.c for ANDROID. basically so ps
>>> can say "bg" or "fg" or whatever rather than an int.
>>
>> In all caps there's just the one?
>>
>>   else if (CFG_TOYBOX_ON_ANDROID)
>>     sprintf(not_o = toybuf+128,
>>             "USER,%%sPPID,VSIZE,RSS,WCHAN:10,ADDR:10,S,%s",
>>             FLAG(T) ? "CMD" : "NAME");
>>
>> The other case insensitive hits are about scheduling policy and a todo about
>> field size?
>>
>> bg and fg... the shell commands?
> 
> no, the background and foreground cgroups used for scheduling policy.
> (the first of the case-insensitive matches you found.)

I need to get an AOSP qemu instance building from source so I can test this stuff...

>> I've missed a curve here.
>>
>>>>>   * ran tests (with some difficulty) on jellybean, and had about 100
>>>>> failures out of about 1200 tests. (`adb shell` didn't return exit
>>>>> status that far back, so this is grep for PASS and FAIL rather than
>>>>> the usual output from my little test runner.)
>>>>
>>>> Statically linked against a newer bionic, but running on the older kernel?
>>>
>>> correct.
>>
>> Seems pretty reasonable then. Do you have a dump of what the failures are so we
>> can categorize them?
> 
> not yet, i was doing it in parallel with a bunch of other stuff. i'll
> have a proper run though now everyone seems in favor of doing this.

Did that ever happen? I've had a crazy month and totally lost context of what I
was doing before...

>>>> You sent a link to https://developer.android.com/about/dashboards which says
>>>> Jellybean is 3.2% of machines checking into the play store or some such. Even
>>>> adding in the 0.6% from the previous 2 releases, that means 96% of all android
>>>> devices are running kitkat or newer.
>>>
>>> aye, but our self-imposed albatross is that we wait until the share is < 1%.
>>
>> Works for me. (And technically the 7 year horizon agrees with you. :)

So the jellybean ABI will still be in the NDK until J drops below 1%, and the
static toybox binary on developer.android.com should be built with the _oldest_ ABI?

>>>> While the 7 year horizon is a bit over Kit Kat's introduction 6 years and 1
>>>> month ago, historical static binary thing is a new feature. I'm ok anchoring it
>>>> to kitkat if you are. That's still going back 2 releases before toybox was
>>>> _introduced_.
>>>
>>> ...which is exactly why those devices need a static toybox the most :-)
>>>
>>> i wish i had a good idea of "oldest release anyone's doing any serious
>>> testing on", but i don't. (and like i say, the shell and adb are awful
>>> that far back too. i don't think adb was fixed until N :-( )
>>
>> I'm in the process of writing a proper shell you can |& with, which can handle
>> its own job control and such. I have:
>>
>>   $ cat boing |& sed s/boing/walrus/
>>   cat: walrus: No such file or directory
>>
>> In my test pile. Alas, it is by _no_ means there yet, but I am at least working
>> on it. :)
> 
> the shell's actually the easy part: it's easier to package that up
> than a static toybox (because fewer dependencies, and especially not
> that awkward libcutils one). i pity anyone who has to do any automated
> testing on old releases. (this is why the world is full of weird stuff
> like parsing ls(1) output to test whether files exist... <movie
> trailer voice>i a world where you can't rely on exit status...)

Circa 2006 reiserfs didn't set d_type in getdents() so I had to change something
in busybox to do an extra stat(). Only filesystem that _ever_ got that wrong, as
far as I know...

(Before I knew the context I mentally pronounced that re-i-serfs, and after the
trial I went _back_ to mentally pronouncing it that way. I don't think I've ever
named a software project after myself, and even Linus tried to call Linux
"freax", it was sysadmin Ari Lemmke who gave him an ftp directory called "linux"
to publish it in...)

>>>> I'm happy to do that, and from the website's nav bar. And yes, I need to catch
>>>> up on patches this weekend, that's one in the stack for this release.

[a month later, still no release out. Yesterday I plugged toybox into the old
aboriginal linux build again and this time I got most of the way through the gcc
build, at which point the tar "no ../ files outside this directory" check is
false positiving on "./" for some reason, gotta track that down. All the corner
cases, gotta constantly regression test, old builds are a good way to do that.
They test stuff I'd never think off...]

>>> it's certainly easier than folks trying to build toybox themselves...
>>
>> I need to get the FAQ updated with build instructions for various targets. (It's
>> in the README, but more docs doesn't hurt...)
> 
> toybox is an example in the NDK docs,

For the old https://developer.android.com/ndk/guides/standalone_toolchain
toolchain type with the python script. The new one at
https://developer.android.com/ndk/guides/other_build_systems uses libpng and
bzip, but the principle's the same. :)

I need to add a FAQ entry on building toybox with the NDK, I have to work it out
again each time I do it...

> but i'm not sure how many people
> actually build it themselves rather than just suffer whatever happened
> to be on whatever device they have.

Given how many people are still sticking busybox on devices, I'd expect there's
a demand for newer toybox binaries on older systems. (Especially if I get the
rest of the 1.0 roadmap sorted.)

Rob