[Toybox] [landley/toybox] Help building toybox with the NDK/bionic (#43)

Fri Dec 30 14:32:55 PST 2016

On Fri, Dec 30, 2016 at 12:46 PM, Rob Landley <rob at landley.net> wrote:
> On 12/27/2016 02:49 PM, enh wrote:
>> On Tue, Dec 27, 2016 at 1:21 AM, Rob Landley <rob at landley.net> wrote:
>>> I'm trying to improve upon "Elliott will tell me if I screwed up the
>>> android build", which seems impolite somehow...
>>
>> although i'd rather triage Android build issues myself rather than
>> never hear of them because of dodgy workarounds committed upstream,
>> which is what happens in some other projects!
>
> True, but "works on musl and glibc, doesn't work on bionic" is the kind
> of thing I generally just poke you and make puppy eyes about anyway.
>
> I'm thinking more along the lines of "finding the sed problem earlier
> would have been nice".

wasn't that actually an LP32 problem?

>>> Long-term I'm hoping to make it easier to build system images. (I'm
>>> aware installing them on actual hardware with "verified boot" is its own
>>> can of worms, ala http://www.youtube.com/watch?v=OfWFmhucWlg but
>>> building and running under qemu should be easier than I currently find it.)
>>
>> or you can just flash your own AOSP -eng build to a Nexus device, "adb
>> root" and do what you like.
>
> I thought they'd renamed it Pixel?

renaming the teams/branches internally and fixing everyone's muscle
memory is a lot harder than changing the marketing label. but, yes,
for the next few years any time i say "Nexus" assume i said
"Nexus/Pixel".

> I've got a Nexus 5. It maxed out at Android M and you stopped shipping
> security updates for it in October. I'm reluctant to buy another device
> that may be discontinued before I get it home.

Nexus 5 was 32-bit only too, so any testing on that would be limited
to the ABI that's -- for platform code -- pretty much dead.

> (The whole "two year support horizon" combined with the implosion of
> cyanogenmod kinda undermines my whole "if you're selling a billion
> phones a year then there are a billion _old_ phones a year available for
> the next generation of developers to play with and learn on instead of
> just landfilling them" theory. Making it easier to build and install new
> system images might work around that, but there's a lot of work to do to
> get from here to there. I need to figure out how to carve out more time
> to work on this again. Endless travel between time zones got me out of
> the "set the alarm at 5am and get a couple hours in before work" habit...)
>
>> qemu is... pretty awful.
>
> It's slow, but using it I can script automated regression tests for a
> half-dozen architectures from a coffee shop without having to plug extra
> hardware into my netbook.
>
> The best part is since an emulator image starts in a known state, the
> mount or ifconfig tests failing to clean up after themselves (or the ps
> tests --bind mounting test data over /proc and getting interrupted
> halfway through so it didn't unmount it) don't screw up the next round
> of tests in a new emulator instance.
>
>> you _can_ use host bionic to run stuff on a regular Linux box, but
>> none of the "interesting" stuff (the stuff in toys/android/) will work
>> there anyway.
>
> Hence booting an android kernel under qemu. But that's something I need
> to work up to...
>
>>> I was thinking that the NDK is the thing changing right now, so it would
>>> get a new symbol to indicate the build environment it provides differs
>>> from the existing AOSP. But you know your build system design better
>>> than I do.
>>
>> it's the other way round: you tell the NDK what API level you're
>> targeting. (because you might legitimately want to say "don't let me
>> use anything that wasn't available before ICS, because i want to run
>> on everything >= ICS".)
>
> Ah, selectable API levels. Hmmm.
>
>>>>> Yay patch. I look forward to it.
>>>>
>>>> sent.
>>>
>>> $ git am 0001-Switch-to-bionic-sys-system_properties.h.patch
>>> Applying: Switch to bionic <sys/system_properties.h>.
>>> error: patch failed: toys/android/getprop.c:17
>>> error: toys/android/getprop.c: patch does not apply
>>> error: patch failed: toys/android/setprop.c:17
>>> error: toys/android/setprop.c: patch does not apply
>>> Patch failed at 0001 Switch to bionic <sys/system_properties.h>.
>>>
>>> The last commit toy toys/android that wasn't from you was on August 4th?
>>> Hmmm... I removed the #ifdef __ANDROID__ back in commit 5b493dc48db0 in
>>> 2015. (And then fixed a screwup in in e5fb6a28ffb0 the next day.)
>>>
>>> We appear to be out of sync? (Speaking of "Elliott will tell me if I
>>> screwed up the android build..." :)
>>
>> that kind of rings a bell, that you did something years ago that
>> caused a merge conflict in toys/android/ and i just skipped it. it
>> used to be useful to me that i could build the Android configuration
>> on non-Android, but i gave up fighting you and now i have the opposite
>> problem to you: i don't/can't test the Android toybox build against
>> glibc.
>
> I'm attempting to fix this, but being able to build an android version
> myself is a prerequisite to me being able to make both work. :)
>
>> new patch attached.
>>
>> you still won't be able to build getprop.c with the NDK, though,
>> because its use of libselinux is quite ingrained.
>
> It's still an improvement. Thanks.
>
>>> So a patch on this would cc: Rich Felker, David Woodhouse, David
>>> Howells, James Bottomley, Doug Gilbert, and Christoph Hellwig. Plus
>>> maybe Al Viro and Andrew Morton.
>>>
>>> But the first part would be coming up with the suggested patch, which
>>> involves comparing musl's and bionic's version of these files (bonic's
>>> at
>>> https://github.com/android/platform_bionic/tree/master/libc/kernel/android/scsi
>>
>> (that's not us. https://android.googlesource.com/platform/bionic/ is us.)
>
> It claims to be a mirror of that, but
> https://android.googlesource.com/platform/bionic/+/master/libc/kernel/android/scsi/
> then.
>
>>> which has a fourth (scsi_proto.h) which git log in the kernel sources
>>> says was introduced in 2015 by Bart an Assche and Hannes Reinecke (never
>>> heard of either of them)...) And then compare them with the kernel
>>> version, try to come up with a set of changes, and cc you and Rich on
>>> the patch before sending it to the list...
>>
>> ours just are the kernel files, scrubbed by machine. actual source
>> (and readme) here:
>>
>> https://android.googlesource.com/platform/external/kernel-headers/+/master/original/
>
> I wonder what packages actually _use_ these headers? (Real world testing
> of the headers proposed for new deployment would be nice...)

the internal code search says 12k files #include linux/...

<linux/input.h> looks to be popular, but there's a lot of stupid shit
too. a surprising number of C programmers have a bad habit of
preferring <linux/whatever.h> over <sys/whatever.h> or <whatever.h>,
which i've never really understood.

>>> Why does spending time working through my todo list never result in a
>>> shorter todo list?
>>>
>>>> the uapi headers aren't great for what i
>>>> assume is their intended purpose. they're missing stuff that should be
>>>> exposed to userspace, still include stuff that shouldn't, don't map
>>>> well to the should-be corresponding POSIX headers, et cetera.
>>>
>>> There are corresponding posix headers?
>
> I checked, susv4 doesn't include the word "scsi" in the tarball I
> downloaded.
>
>> what i mean is cases where you have <sys/whatever.h> #include
>> <linux/whatever.h>. most of the time you get too much or too little
>> (or both at the same time).
>
> The musl headers have a policy of not ever using/including the linux/*
> headers at all, and I had to make puppy eyes at Rich to have
> musl-cross-make even install them.

that would certainly discourage the anti-social from including all
kinds of random <linux/...> crap.

> (I fought the "cut and paste the linux header bits into your own code"
> wars back on busybox, from
> http://lists.busybox.net/pipermail/busybox/2004-November/013128.html to
> http://lists.busybox.net/pipermail/busybox/2010-March/071874.html and
> the answer was "don't go there if you want to support more than one
> architecture".)
>
>>> Yes, but the clang wrapper isn't prefixed. If I'm building for mips it
>>> distinguishes host toolchain from target toolchain with a prefix, not
>>> with compiler flags. (Because that's the way other packages that cross
>>> compile did it.)
>>>
>>> How do you cross compile your kernels with clang, for example? I'm happy
>>> to hit my build with a rock until it's in a new shape, just dunno what
>>> that new shape should look like. Don't want to invent something unique
>>> if I can help it.
>>
>> building kernels with clang is still a work in progress.
>
> I've been following that for years:
>
> http://lists.landley.net/pipermail/aboriginal-landley.net/2011-July/001862.html
>
> I suspect the main reason https://www.youtube.com/watch?v=TiLPL-zMndM
> hasn't made much progress in the 3 years since that video is they
> allowed the Linux Foundation to get involved. When the money ran out,
> http://llvm.linuxfoundation.org stopped updating. (Most recent status
> I've seen is
> https://www.phoronix.com/scan.php?page=news_item&px=Clang-The-Kernel-2016)

i think the "we aren't using gcc any more, and 4.9 was the last
version we'll update to, and we're not maintaining that any more
either" message is pretty well understood at this point, so several
different groups are independently working on this, for different
architectures. the google folks seem to have gotten further than the
linaro folks judging by conversation at plumbers this year, but i
think it's clear to everyone that it's a priority at this point.

> But my opinion of the Linux Foundation hasn't changed since
> http://landley.net/notes-2010.html#18-07-2010 and was more or less
> confirmed by Eben Moglen's explanation at
> https://www.softwarefreedom.org/blog/2016/apr/11/lf/ about how the Linux
> Foundation is a "trade association" and not a 501c3.
>
> I worked on building the kernel under another compiler myself ten years
> ago, when I maintained my own tinycc fork and tried to build the kernel
> with it. Didn't get too far (some ELF tool I hadn't implemented yet like
> objdump or nm was parsing a .o file to produce ascii output that was run
> through sed or awk to produce a header file with constants for structure
> offsets). But it wasn't that hard, either.
>
> Fabrice Bellard got tinycc to build a bootable linux kernel in tccboot
> back in 2004, so we know it's doable. (Ok
> http://blog.smartbear.com/careers/fabrice-bellard-portrait-of-a-super-productive-programmer/
> is not a good yardstick for mere mortals but I was doing ok at it, the
> _work_ was all on the compiler side with tinycc still missing a bunch of
> basic features like constant propagation and dead code elimination,
> stuff either required by c99 or present back in Turbo C for DOS. If
> you're willing to push patches upstream into the kernel instead of
> laboriously improve your tools to handle the existing kernel code, it
> gets a lot easier.)
>
> The trick is to start with a minimal config and add features. I'm told
> they got a minimal config building ages ago, so defend it with
> regression testing each release and work on new features one at a time...
>
>> but i'm not
>> sure what you're talking about. make-standalone-toolchain generates
>> little wrapper scripts for exactly this:
>>
>> /tmp/arm64-unified$ cat ./bin/aarch64-linux-android-clang
>
> Ah, I'd missed that. (I just saw the "clang*" in that directory. Hmmm...
> CC=clang CROSS_COMPILE=aarch64-linux-android- should work then. Thanks,
> I'll give it a try once I carve out some more toybox development time.
>
> (Email can be composed in 2 minute increments. Coding requires a bit
> more sustained focus, for me at least...)
>
>>>> and i'd still like a
>>>> hermetic build by having a toybox-linked-with-host-bionic for use in
>>>> the build itself, replacing all the stuff that usually comes from the
>>>> host /bin.
>>>
>>> See "make install_airlock".
> ...
>>> http://lists.landley.net/pipermail/aboriginal-landley.net/2016-November/002591.html).
>>>
>>> It doesn't support weight yet, but I'm working on it. Still a week to
>>> New Year's...
>
> See $DAYJOB Perpetual Crisis last message. This got hit too. :(
>
>>> Under AOSP it's "select your hardware platform" and then it builds a
>>> thousand packages for it and I haven't figured out how to tell it NOT
>>> to. I'd like to break that down into chunks a bit, starting with "build
>>> a base android system that boots to a shell prompt under stock QEMU"
>>> (like the embedded world's doing anyway), then "add android's init, the
>>> system manager, and all that SELINUX stuff", then maybe enough to run a
>>> single fullscreen Java app (which I think ChromeOS factored out?
>>
>> not afaik. i think they just run lightly-modified "full" android in a container.
>
> So a todo item then.
>
>>> There's
>>> a natural "everything spawned from zygote" layer but I'm not sure where
>>> that starts) and then the full desktop stuff. (And then the Play Store
>>> can of worms, which the rest of the world has zero visibility into China
>>> and India are ignoring anyway according to Clay Shirky's interview about
>>> his year teaching at NYU's Shanghai campus at
>>> http://supchina.com/sinica/hour-clay-shirky/)
>>>
>>> I.E. Orthogonal Layers would be nice. But last I checked, tackling that
>>> is a huge undertaking. And the "make a web server or router or build
>>> machine or drone flight system or 3d printer using android plumbing"
>>> layer isn't there at all that I can find.
>>
>> brillo always includes Java now it's s/brillo/things/:
>> https://developer.android.com/things/sdk/index.html
>
> I await Oratroll's lawsuits:
> http://www.businessinsider.com/oracle-starts-to-audit-java-customers-2016-12
>
> (Does this have an AOSP build target? Ironically, the new name is
> _slightly_ less easy to google for.)

they're still in a separate manifest.

>> one prerequisite to a secure IoT is
>
> Airtight container infrastructure?
>
>> that everyone stops shooting at
>> the nuclear missiles^W^W^W^W^Wwriting unnecessary native code.
>
> You mean like qualcomm's baseband binary blob?

http://bits-please.blogspot.com/

> [deletes 100 line rant]
>
> How should I phrase this... "I'm not sure your experience here is
> universal."
>
> Rob

-- 
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Android native code/tools questions? Mail me/drop by/add me as a reviewer.