[Toybox] Android .config pondering.

[Rob Landley]

On 5/5/22 12:24, enh wrote:
> On Wed, May 4, 2022 at 8:15 PM Rob Landley <rob at landley.net> wrote:
>>
>> So I thought there should probably be a "make android_defconfig" target even if
>> android isn't using it, just so I have an easier time tracking stuff. Except
>> android has three configs: device, linux, and mac.
> 
> meh, like i always say, i think it's a sensible division of
> responsibilities for you to worry about "does it work on debian?" and
> android (and to a lesser, more "good guy greg" extent, mac) is my
> problem?

Oh sure. I occasionally find myself asking the question "is this command enabled
on android" and then I have to go fish for it, but my main motivation for
looking at this is I'm doing documentation videos and trying to explain stuff
highlights holes. The default defconfig is basically linux_defconfig, and then
there's make macos_defconfig and make bsd_defconfig... but no make
android_defconfig. Seemed less awkward to just add one than explain why it's not
there. My macos_defconfig isn't what AOSP builds for mac either. :)

>> The android .config-mac the output of "make macos_defconfig" differing seems
>> like a "me" problem (I should update my macos_defconnfig, and ideally I'd get an
>> ssh into a mac box so I can see what actually builds and passes its tests. I
>> could kinda hack the github build stuff in a fork to do that maybe? Rather a lot
>> of iteration required though, unless maybe "make change" works on mac...)
> 
> iirc the reason they differ is that macos_defconfig is "everything
> that passes the tests on a mac [the last time enh ran all the tests on
> a mac]" whereas the android mac config is basically "the intersection
> of macos_defconfig and android's _linux_ config".

The android one had a couple commands that weren't in macos_defconfig, so I
added them. :)

>> But the other two are... kinda confusing? The device (android) config has a lot
>> of stuff the linux (build) one does not: acpi, base64, blkdiscard, blkid,
>> blockdev, cal, chattr, chcon, chgrp, chown, chroot, chrt, cksum, clear, devmem,
>> df, dmesg, expand, fallocate, false, file, flock, fmt, freeramdisk, free,
>> fsfreeze, fsync, getenforce, getfattr, groups, gunzip, help, hwclock, i2c*,
>> iconv, ifconfig, inotifyd, insmod, ionice, iorenice, iotop, kill, killall,
>> load_policy, logname, log, losetup, lsattr, lsmod, lsof, lspci, lsusb, makedevs,
>> mkfifo, mknod (twice?), mkswap, modinfo, modprobe, more, mount, mountpoint,
>> nbd_client, netcat, netstat, nice, nohup, nsenter, partprobe, pidof, ping,
>> ping6, pivot_root, pmap, printenv, pwdx, readelf, renice, restorecon, rev,
>> rfkill, rmmod, rtcwake, runcon, sendevent, setenforce, setfattr, sha224sum,
>> sha384sum, split, strings, stty, swapoff, swapon, sync, sysctl, tac, taskset
>> (twice), time, top, traceroute, tty, tunctl, uclampset, ulimit, umount, unlink,
>> unshare, uptime, usleep, uudecode, uuencode, uuidgen, vconfig, vi, vmstat,
>> watch, and yes.
> 
> yeah, so for "android" my policy (to the extent there's any kind of
> policy!) is basically "does it pass the tests on android?". if it
> does, meh, ship it! someone might find it useful.

Woo! I'm all for it.

> there's a slightly
> higher bar for whether or not it gets a symlink (which is actually a
> separate list in the .bp file) --- but that's a lot more vague,
> something like "does enh think this is either (a) in a good enough
> state that we can realistically support it effectively or (b)
> sufficiently useful that 'half an eye is better than no eye' and we
> should make it easily available anyway, and deal with issues as they
> arrive?".

If it's not in pending, send me the bug reports.

(I'm simultaneously happy somebody is looking at sh.c and sending me patches,
and going "it's not ready for company..." The next test_sh that fails is the one
checking that pipeline segments are properly backgrounded, because right now if
you pipe a while loop into another while loop the second one won't start running
until the first one finishes because they're happening int the same shell
context. One of the largeish reworks I got distracted from many moons ago is
most of the way to making that work, but it's nontrivial...)

>> In fact only two symbols are enabled in the linux host but NOT the device:
>> CONFIG_TOYBOX_LSM_NONE just means selinux support was NOT selected, and
>> CONFIG_TOYBOX_SUID... can't work in the AOSP build? (Not only does the android
>> build not run as root, but it doesn't CLONE as root either, meaning the toybox
>> binary can't be installed suid root, so never has any privileges to drop.)
> 
> those will just be historical accidents. i can't actually remember
> whether i started the two host configs from the android config, or
> from separate defconfig/macos_defconfig runs. (there's probably stuff
> in the files that doesn't exist any more, like the recent CATV
> removals. a script to check for such things wouldn't be useless :-) )

$ cat blah.sh
#!/bin/bash

x() {
  sed -n 's/.*\(CONFIG_[^ =]*\).*/\1/p' "$1" | sort
}

egrep "^($(diff -u <(x "$1") <(x "$2") |sed -n 's/^+C/C/p' | xargs | tr ' '
'|'))" "$2"
$ ./blah.sh ~/toybox/clean/.config .config-linux
CONFIG_TOYBOX_I18N=y
CONFIG_CAT_V=y
CONFIG_CP_PRESERVE=y

>> I guess this means I should use the device .config?
> 
> if your aim is "build and test what android ships on the device", yes.

That's more or less what I'm trying to track with this exercise.

> if your aim is "build and test what android uses on the host to
> _build_ android", you want the linux config.

I'm testing toybox defconfig for the build environment stuff.

> (and if you're really not-so-secretly interested in "what i'd need to
> build android on android", you'd still just need the linux config :-)
> )
> 
>> (It sounds like what
>> .config-linux wants is everything built but only certain symlinks populated?
>> Which I guess is a dependency tracking thing?)
> 
> no, historically the linux config is "what does the AOSP build need?".
> that's a much smaller set than "full" toybox.
> 
> that said, i am quite tempted to just add "all the stuff from posix/"
> to the linux set. but the rate at which i get asked to add missing
> stuff (usually by the folks building the _kernel_ than folks building
> AOSP, because AOSP is locked down by default, and you can't just
> "accidentally" add a dependency on a new host tool) is so low that
> it's just not been worth it.

I point you at PENDING= in scripts/install.sh, which is everything I've needed
for mkroot.sh to build kernels for:

https://landley.net/toybox/downloads/binaries/mkroot/latest/

The relevant section is:

# The following are commands toybox should provide, but doesn't yet.
# For now symlink the host version. This list must go away by 1.0.

PENDING="dd diff expr git tr vi bash sh xzcat bc ar gzip   ftpd less awk unxz
bison flex make nm"

# "gcc" can go away if the kernel guys merge my patch:
# http://lkml.iu.edu/hypermail/linux/kernel/2202.0/01505.html
TOOLCHAIN="as cc ld gcc objdump"

> (and especially not worth dealing with
> the can of worms that is "aye, but you only have one directory of
> symlinks for prebuilts/build-tools and if you add something there
> 'just because' then it [aiui] automatically gets made available for
> build rules, which is probably something i shouldn't just unilaterally
> do" :-) )

Back in the day, every busybox command I added to aboriginal's linux from
scratch build I had to rebuild all the packages to see if any of the configure
steps would decide to go crazy.

You at least have much less gnu autoconf exposure to deal with. (I hope.)

>> Rob

Rob

P.S. Because the powerpc64le build calls "bash" out of the $PATH rather than as
#!/bin/bash, that's why it needs the bash symlink in the $PATH. Only
architecture I've seen do that so far...

P.P.S. I don't remember why vi and diff need to be in the $PATH, I can try the
builds without them and see...