[Toybox] building multiple configurations from one tree

[enh]

On Fri, Jun 30, 2017 at 12:52 PM, Rob Landley <rob at landley.net> wrote:
> On 06/30/2017 01:13 AM, enh wrote:
>> i've mentioned this a few times, but i have a concrete example now.
>> with treble [https://android-developers.googleblog.com/2017/05/here-comes-treble-modular-base-for.html]
>> there will be a copy of mksh and toybox on the /vendor partition in
>> addition to the copies on the /system partition. this lets us upgrade
>> /system's copy at will without worrying about breaking any of the code
>> on /vendor (because it'll only be allowed to access the /vendor shell
>> and utilities).
>>
>> so far so good. and ideally you'd want them to be the same. but the
>> functionality and libraries available to /vendor code differ from
>> /system code. in particular, some of the Android extensions to SELinux
>> aren't available for /vendor. which basically means getprop -Z and
>> restorecon can't be built.
>
> Which sounds like the "building with the NDK" problem all over again.

yes and no. the NDK problem is fixable within your current system
(it's just a low priority for me because i'm assuming what we _really_
want is to be able to build a toybox binary that works as well as
possible even when run on older devices, and doing that right requires
NDK work that we're not likely to get to before 2018).

this is more like "what if you needed to build for the NDK and glibc
in a world where you have to just run the scripts to generate files
once".

>> short term we could provide dummies in portability.h:
>>
>> #if __ANDROID_VENDOR__
>> static inline int f(...) { error_exit("f unimplemented on /vendor"); }
>> static inline int g(...) { ... }
>> #endif
>>
>> and then -D__ANDROID_VENDOR__ when we build the /vendor copy.
>>
>> but i think it would be better if we just don't have restorecon and getprop -Z.
>>
>> i can send a patch that makes getprop -Z require a new "has Android
>> SELinux extensions" configure variable, but there's still no easy way
>> for me to override stuff in the .config file (which would let me
>> control both this and the availability of restorecon).
>
> I can add a compile time probe, but you're not using my build scripts.

you know this already, but just to be clear for those following along
at home --- i am using your scripts; i'm just not using them at build
time. i run them once every time i sync, and then check in the
resulting generated files.

> I'd also point you at miniconfig
> (http://landley.net/aboriginal/FAQ.html#dev_miniconfig) but again...
>
> (Sigh. I designed infrastructure for this sort of thing.)
>
>> if config.h instead of
>>
>> #define CFG_RESTORECON 1
>> #define USE_RESTORECON(...) __VA_ARGS__
>>
>> said
>>
>> #ifndef CFG_RESTORECON
>> #define CFG_RESTORECON 1
>> #endif
>> #if CFG_RESTORECON == 1
>> #define USE_RESTORECON(...) __VA_ARGS__
>> #else
>> #define USE_RESTORECON(...)
>> #endif
>>
>> and global_union could have
>>
>> #if CFG_RESTORECON
>>         struct restorecon_data restorecon;
>> #endif
>>
>> i could -DCFG_RESTORECON=0 (and not have restorecon.c in the files
>> list) when building the vendor copy.
>
> True. Not an ideal solution. And roughly equivalent to hitting config.h
> with sed. There's no dependency resolution going on there, so might as
> well just:
>
> sed -E '/ (CFG|USE)_RESTORECON /{s/1$/0/;s/__VA_ARGS__//}' \
>   generated/config.h
>
> But again, not an ideal solution. :)
>
>> i have another couple of configurations lurking in the wings too.
>> they're likely to not have any SELinux, and might want to support
>> different subsets of commands.
>
> Grrr. I thought you could feed a KCONFIG_ALLCONFIG to defconfig, but
> looking at the kconfig source (copied from 2.6.12 back in the day, I've
> meant to replace it _forever_) it's only reading it for
> allno/yes/mod/randomconfig. Sigh.
>
> And even if that did work it wouldn't help you because you've
> snapshotted the generated files.

and because i might be building on a Mac host (don't blame me), even
if your scripts did let us say "dump the generated stuff in directory
X" so we could do it as part of the build, that just seems like asking
for other [Mac portability] problems.

> Right, I need to redo my build so it's something you can actually _use_,
> and step 1 of that is getting the existing Ninja build to run the
> Android.mk file you've got. But "apt-get install ninja-build" in Ubuntu
> 14.04 was followed by:
>
>   $ ninja -f Android.mk
>   ninja: error: Android.mk:17: expected '=', got ':'
>   LOCAL_PATH := $(call my-dir)
>              ^ near here
>
> Which put it back on the todo list. :)
>
> Then again Ubuntu 14.04's "aptitude show ninja-build" says:
>
>   Homepage: http://martine.github.com/ninja/
>
> Which is 404 (it appears to be ninja-build.org now) so that's possibly a
> really old version. I should try building it from source...

i doubt you'll get very far without a full AOSP tree. as we transition
from make to soong, there's a lot of chewing gum and string holding
everything together.

>> i think previously you've been thinking along the lines of making it
>> easier to work with multiple generated/ directories, but i'm wondering
>> whether it might be more convenient to just generate something that
>> can be overridden up front? (it would certainly be a good fit for my
>> uses.)
>
> Both need to happen, but you've got a use case sitting in front of you
> right now so let's fix that.
>
>> i haven't actually tried this yet, so maybe there's something that
>> won't work. but if this seems like a reasonable thing to try i can
>> have a look. i'll probably have a look anyway, because having multiple
>> copies in the tree or having local #if __ANDROID_VENDOR__ patches seem
>> like significantly worse choices and i don't want to be without an
>> alternative when the music stops and we have to sit somewhere...
>> though dummy implementations is probably the smallest hack if it comes
>> to it.
>
> 90% of my toybox work is on the weekends these days. (Startup $DAYJOB
> has missed 2 of the last 3 paychecks so I'm currently doing a consulting
> gig on the side of that, which means my weekdays are pretty thoroughly
> shot right now. I'm just by fiat saying "weekends are for toybox!" and
> ignoring 'em both during that).
>
> Which means I get 2 days to poke at this now, if you think it's worth
> trying to fix it up.
>
> The current problem seems to be that your build system is using a
> snapshot of the generated directory checked into your repo, and now you
> need multiple build contexts with different configurations. We could
> hack up those saved configs with more #ifdeffery, or hit them with sed,
> but long-term getting the generated/ stuff generating under your build
> seems like a better solution? I'd like to at least get _started_ on that.

well, like i said... macOS makes this likely to just be a different headache.

but, yes, if there was a script that i could say "take .config file X
and write all the generated stuff to directory D", i think we can
plumb the rest in.

> Which means I probably need to reproduce your build context. I could
> reinstall AOSP on the big machine (which is nontrivial because if I
> recall I need to clear off something like 250 gigabytes of disk space,
> and it currently has 4 gigs free).

i don't think you do. just have two .config files in a regular toybox
checkout, and be able to build them both.

you can see what i currently have at
https://android.googlesource.com/platform/external/toybox/+/master/Android.mk

i'll still have to manage multiple lists of files in any multi-config
world, but that's fine: it just gives every instance motivation to
minimize unnecessary differences. and realistically i don't see it
ever being anything other than stuff like "do i have SELinux?", "do i
have the Android extensions to SELinux?", which is why i think letting
me -D in my cflags is probably the best for me.

(as i've said, macOS gives me mixed feelings about generating the
generated files at runtime, plus it feels like that road leads to us
duplicating more of your infrastructure in our build system [as we
have to do things like build the C help generator first and so on].)

> But the BIGGER problem is last I poked at that it was a giant monolith
> that was non-obvious how to focus on individual bits of it. (I had a
> series of web pages to read about that, and a youtube presentation. It's
> on the todo list. But those resources were for "M" and we're up to "O"
> now, so I expect a lot's changed anyway...)
>
> Ideally I'd like to reproduce the android build context in an otherwise
> minimal chroot, but I have no idea how to _start_ going about that.
> Other than installing, successfully running, and then reverse
> engineering AOSP.
>
> Rob

i did experiment this afternoon with making the getprop -Z
configuration more specific (that there are Android SELinux
extensions), and i changed generated/config.h to be more like this:

#ifndef CFG_GETPROP_Z
#define CFG_GETPROP_Z 1
#endif
#if CFG_GETPROP_Z
#warning with getprop_z
#define USE_GETPROP_Z(...) __VA_ARGS__
#else
#warning without getprop_z
#define USE_GETPROP_Z(...)
#endif

the only other thing i had to do (which i did manually because it
wasn't as obvious how to fix this) was generated/flags.h:

#if CFG_GETPROP_Z
#define OPTSTR_getprop ">2Z"
#else
#define OPTSTR_getprop ">2"
#endif

unlike config.h and globals.h which are trivial, it wasn't obvious to
me how to fix mkflags.c.

but with the generated files modified as described, i could just add
-DCFG_GETPROP_Z=0 in the vendor toybox's cflags and otherwise reuse
everything.

-- 
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Android native code/tools questions? Mail me/drop by/add me as a reviewer.