[Toybox] [landley/toybox] Help building toybox with the NDK/bionic (#43)

[enh]

On Wed, Aug 17, 2016 at 11:28 AM, Rob Landley <rob at landley.net> wrote:
> On 08/16/2016 04:47 PM, enh wrote:
>> On Tue, Aug 16, 2016 at 2:22 PM, Rob Landley <rob at landley.net> wrote:
>>> I have a machine I can clear 200 gigs off of, but it's not the netbook I
>>> have with me most of the time. Is there any way I can build and then
>>> _export_ a toolchain from AOSP?
>>
>> i wasn't kidding when i said that making the NDK headers sane is a
>> work in progress :-) here's a patch from 4 hours ago:
>> https://android-review.googlesource.com/#/c/256800/
>
> I've maintained http://landley.net/aboriginal/about.html (under various
> names) for about 15 years. Trust me, I know the feeling. :)
>
>> historically the NDK headers were a _fork_ of the bionic headers.
>> we're working on unforking, and working on making it so that no matter
>> which version of android you're targeting (gingerbread, say), you're
>> still using up-to-date headers; they'll just not make available stuff
>> that wasn't in gingerbread. (if you're using clang, you'll even get
>> error messages explicitly telling you "f was introduced in jellybean
>> but you're targeting gingerbread" rather than just "implicit
>> declaration of f" or whatever.)
>
> Ooh, snazzy.
>
>> so there's no existing "update the headers" process/scripts, just the
>> stuff we're working on right now.
>>
>>> Presumably the NDK comes from somewhere...
>>
>> yeah, it's just another manifest (master-ndk rather than master) in
>> AOSP. instructions to build it here:
>> https://github.com/android-ndk/ndk
>>
>> but like i said, it's not that we have a fixed NDK and just haven't
>> shipped it yet --- we're still working on fixing the headers.
>>
>> but getting back to your intent...
>>
>> it's a pain in the ass, but if you have an AOSP build and an unpacked
>> NDK (or an NDK you built yourself; doesn't matter), you can copy the
>> libc.so and libc.a out of out/, and you can cp -r the headers out of
>> bionic/. the awkward part with the latter is flattening the uapi
>> headers. in the platform they're kept nice and separate (and
>> per-arch), but in the NDK they end up all collapsed into a regular
>> sysroot.
>
> So the solution is grab AOSP and harvest its organs. Ok.
>
>> alternatively (though i've never done this), i don't see why you
>> couldn't point your NDK compiler at the relevant lib and include
>> directories in the platform.
>
> I wanted to use an official NDK so I saw the same bugs as you, but this
> is apparently not a thing.
>
> My next choice would be to build bionic and staple it to an existing
> toolchain, but it doesn't offer a standalone build option and only
> provides makefiles for use withing a giant integrated monolithic build.
>
> So my _third_ choice is to grab files out of that giant integrated
> monolithic build, put them in an empty directory at a different path,
> and hit them with a rock until they play nice. (My preferred rock for
> this is ccwrap.c.)
>
>> but the easiest for those with enough disk space...
>
> Literally half my netbook's disk.
>
> Sigh, I can get a terabyte ssd and do surgery to the netbook, I just
> tend to lose multiple working days to that sort of thing because it's
> never simple.
>
> (Last time I bought a new netbook to replace this one, Ubuntu couldn't
> make the sucker suspend to ram. Same manufacturer, similar model just a
> newer version, multiple people tried to make it work for multiple weeks,
> but in the end we gave up. Could be worse: Fedora couldn't drive that
> box's touchpad right. Linux on the Desktop in 2016: still sort of almost
> thereish.)
>
>> if you check out an AOSP tree, change the external/toybox/Android.mk
>> to say "LOCAL_FORCE_STATIC_EXECUTABLE := true" and then mma -j in that
>> directory, that should leave you with a tree with just enough built.
>
> Last time I tried this just the _checkout_ was something like 30 gigs.

make sure you sync with "-c" to not get the history.

> I'm in San Diego at the moment with a 4 gig monthly phone tethering
> limit (thanks t-mobile!) and ~120k/second coffee shop wifi.
> (30*(1<<30))/(120000) is 268435 seconds, which is 4473 minutes, which is
> 74 hours. (To download the NDK tarball, I sat at a McDonalds until it
> closed and then sat outside at the concrete tables another 45 minutes.)
>
> I'll see what I can do. Maybe I can find a faster coffee shop. Or wait
> until I get back to texas. (Looking like "february".)

if you can wait until february (and i'm sure you have enough other
stuff to do in the meantime), this particular problem should have
solved itself. we already know toybox can be built with the platform
headers because it's checked in to the platform. afaics, the only bug
here is "the NDK headers are three years out of date", and that's our
bug, not yours :-)

> (Just did an hour and a half of downloading on the fastest internet I
> could find. 1.7 gigs. I'll resume later, I guess...)
>
> Four more hours of downloading got it up to 3 gigs, according to df -s.
> I'm not sure how much of that's useful though. Did you know that if one
> of the git repositories it's downloading is 200 megs into and you hit
> ctrl-c, next time it starts downloading that from 0% _but_ the size
> usage according to df does not go down? Given that I've had to interrupt
> it 3 or 4 times already, I'm guessing there are several hundred
> megabytes of wasted space on this, and no way to run "git gc" or similar
> that I've found to get it to _notice_ said wasted space.
>
> Oh well, here's hoping 30 gigs of space is enough to download everything
> and then copy the repository to the larger machine that isn't easily
> portable out to non-tethered internet but has a terabyte disk in it I
> might be able to clear enough space off of to actually build this thing...
>
>> i guess you might be able to then prune away enough unused stuff
>> (device/, hardware/, vendor/, frameworks/, art/, and so on) to
>> transplant that onto a smaller disk.
>
> I'll happily extract the compiler out of the tree with pliers and a
> whip, it's getting it to build the thing first that I can run _out_ of
> the tree that's tricky. Hence my interest in the NDK binaries...
>
> Thanks,
>
> Rob



-- 
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Android native code/tools questions? Mail me/drop by/add me as a reviewer.