[Toybox] Building AOSP...

[enh]

On Wed, Jan 14, 2015 at 12:33 AM, Rob Landley <rob at landley.net> wrote:
>
>
> On 01/13/2015 12:02 PM, enh wrote:
>> On Mon, Jan 12, 2015 at 11:35 PM, Rob Landley <rob at landley.net> wrote:
>>>
>>>
>>> On 01/10/2015 05:04 PM, enh wrote:
>>>> On Thu, Jan 8, 2015 at 10:55 AM, Rob Landley <rob at landley.net> wrote:
>>>>> Since Toybox made it into Android, I just reinstalled my big machine
>>>>> with Ubuntu 14.04 so I could actually add packages to it (such as the
>>>>> java version the Android Open Source Project wants), and I have
>>>>> downloaded AOSP (overnight, and I had to restart it in the morning), and
>>>>> it is now compiling. (Wow, that's a lot of warnings...)
>>>>
>>>> upload your patches to https://android-review.googlesource.com/ :-)
>>>
>>> Actually my problem right now is I can't get the emulator to run.
>>>
>>>> landley at halfbrick:~/android/aosp$ emulator
>>>> emulator: WARNING: system partition size adjusted to match image file
>>>> (550 MB > 200 MB)
>>>>
>>>> emulator: WARNING: data partition size adjusted to match image file
>>>> (550 MB > 200 MB)
>>>>
>>>> SDL init failure, reason is: No available video device
>>>
>>> I ran the build under "screen", which meant the session that the build
>>> set a magic environment variable for didn't have access to X11. So I ran
>>> another window that _did_ have an X11 context and that failed because it
>>> didn't have the magic environment variable(s).
>>>
>>> So then I tried:
>>>
>>>> landley at halfbrick:~/android/aosp$ ANDROID_PRODUCT_OUT=/home/landley
>>>> /android/aosp/out/target/product/generic prebuilts/android-emulator
>>>> /linux-x86_64/emulator
>>>> emulator: ERROR: You did not specify a virtual device name, and the
>>>> system directory could not be found.
>>>>
>>>> If you are an Android SDK user, please use '@<name>' or '-avd <name>'
>>>> to start a given virtual device (see -help-avd for details).
>>>>
>>>> Otherwise, follow the instructions in -help-disk-images to start the
>>>> emulator
>>>
>>> So then I started reading -help-disk-images (after a few minutes of
>>> trying --help-disk-images and it not knowing what I was talking about
>>> because normal --longopt syntax does not apply here), and that told me I
>>> needed to manually specify at least 4 images (possibly 9 if you could
>>> the writable .img files), so I started googling and
>>> http://developer.android.com/tools/help/emulator.html told me I needed
>>> to read http://developer.android.com/tools/devices/emulator.html which
>>> told me I needed to read
>>> http://developer.android.com/tools/devices/index.html which didn't
>>> explain how the other context tried to just run it (stymied by X11 not
>>> being connected up, not by being unable to find the build output)
>>> without me needing to create a new thing or specify multiple files
>>> manually on the command line...
>>>
>>> Which is about the point I wandered away and did something else...
>>
>> here's what my shell history says i use if i just want a shell:
>>
>>   emulator -partition-size 1024 -verbose -show-kernel -no-window
>
> $ cd android/aosp
> $ prebuilts/android-emulator/linux-x86_64/emulator -partition-size 1024
> -verbose -show-kernel -no-window
> emulator:Can't determine target AVD architecture: defaulting to arm
> emulator:Looking for emulator-arm to emulate 'arm' CPU
> emulator:Probing program:
> prebuilts/android-emulator/linux-x86_64/emulator64-arm
> emulator:Found target-specific emulator binary:
> prebuilts/android-emulator/linux-x86_64/emulator64-arm
> emulator:Probing for:
> prebuilts/android-emulator/linux-x86_64/libOpenglRender.so
> emulator:Probing for:
> prebuilts/android-emulator/linux-x86_64/lib/libOpenglRender.so
> emulator:Found OpenGLES emulation libraries in
> prebuilts/android-emulator/linux-x86_64/lib
> emulator:Setting LD_LIBRARY_PATH=prebuilts/android-emulator/linux-x86_64/lib
> emulator: ERROR: You did not specify a virtual device name, and the system
> directory could not be found.
>
> If you are an Android SDK user, please use '@<name>' or '-avd <name>'
> to start a given virtual device (see -help-avd for details).
>
> Otherwise, follow the instructions in -help-disk-images to start the
> emulator
>
> [Clearly I have no idea what I'm doing. As usual...]

as you discovered, here too there's no saved state: your choice of
architecture (and hence system image) is based on the same environment
variables used to build. (the "avd" is, aiui, the alternative. if
you're a third-party app developer you probably don't have a built
Android tree, so you point the emulator at an "avd" which _is_ a
stored configuration.)

>> if i build x86-64 i can get native speed via:
>>
>>   emulator -partition-size 1024 -verbose -show-kernel -no-window -qemu
>> -enable-kvm -m 1024
>>
>> (assuming you have kvm enabled, of course.)
>>
>>> I should probably re-run the build so it can set up the magic
>>> environment variables again (last time it took 5 hours), except then I
>>> get one magic window that might run the emulator due to context I can't
>>> reproduce, which seems... awkward. I would actually like to understand
>>> it. (Or just boot the darn thing under vanilla qemu, which
>>> prebuilt/.../emulator seems to be a fork of? I think?)
>>
>> certainly the aim is to get back in sync with upstream, but i've no
>> idea where that stands right now.
>
> If there's a list for it, I'd be interested. I'm slowly poking at qemu
> internals more...

not that i know of.

>>> I think I did build that. What config file stores the "lunch" answers I
>>> gave? (The only dotfile I see in the directory is .repo. There's no
>>> "output" directory. I can't do "git status" in the directory because it
>>> says there's no git repo there. Currently running "repo status" on a
>>> wild guess, but that's spent 2 minutes sitting there without producing
>>> any output yet...)
>>
>> they're transient. stored in environment variables. there's nothing
>> but available disk space to stop you from using the same tree to build
>> arm, arm64, mips, x86, and x86-64 emulators and Nexus 4, 5, 7, 9, 10,
>> and ...
>
> So how does "emulator" know which one I'm trying to launch?
>
> Ah, it doesn't. I have to not just re-source the engsetup.sh file but
> re-run lunch and select the architecture, and _then_ the emulator ran!
>
> And ran.
>
> Three minutes, still no shell prompt. 100% CPU usage (Well, 1 processor).

if you think that's bad, you should try the mips emulator. like i said
(but perhaps not strongly enough) unless you have a really beefy
machine you probably want to build x86 or x86-64 and use the emulator
with kvm. that's as fast as a real device.

> Interesting, scrolling back past the healthd and lowmemorykiller output,
> I see:
>
> logd.auditd: start
> shell at generic:/ $ healthd: BatteryVoltagePath not found
> healthd: BatteryTemperaturePath not found
>
> But hitting return or typing "ls -l /" and enter again doesn't give me a
> console prompt. So there _is_ a shell, but it's not hooked up to my input...
>
> And when I killed the emulator process from another window, the ls -l
> ran in host context after it exited. So the emulator wasn't reading from
> stdin.
>
> Still: progress!
>
>   $ emulator --help
>   unknown option: --help
>   please use -help for a list of valid options
>
> That's nice.
>
> Wait, this is a wrapper around qemu? (Why is the wrapper an executable
> rather than a shell script?)

yes. don't know.

the option you were looking for was probably -shell. but personally i
just let the emulator run in the background and "adb shell" into it.
(though to be clear, like most people i actually almost always use a
real device.)

> Maybe I can just intercept the qemu command line and make things work in
> a context I'm more familiar with. (Well, apparently not a wrapper around
> _stock_ qemu. Is the source to the "prebuilts" in here somewhere? If so,
> why are they prebuilt?)

the source is in the tree, but you already complained about how long
builds take. they'd take even longer with fewer prebuilts :-)

>>> I'm prioritizing "doing it right" for toybox, but I added the "pending"
>>> directory because I'm a huge bottleneck when I do that (cleaning up
>>> printf.c just ate most of a week) and this is a place code can go
>>> without being ignored/lost on the list.
>>
>> what's your long term plan for the bottleneck for patches to things
>> that have already been promoted? (like id or touch, say.) obviously in
>> the very long term this should solve itself as there becomes ever
>> fewer bugs left, but there's probably a decade between now and then
>> :-)
>
> Ooh, you do ask the tough questions.
>
> The ones that are blocked tend to be blocked _on_ something. For example
> I just looked at your chgrp/chown patch and my cop of that is hacked up
> with half an attempt at making scripts/single.sh work on both chgrp and
> chown. (Lemme go finish that, actually...)
>
> Still, I should do a triage pass and at least explain what the issues
> are with each one here on the list. (I tend to see the other issue and
> then follow it and get distracted.)

yes, because otherwise there's no way we can help move our patches
forward, and both sides are blocked. public review is also more likely
to lead to a steady state where we converge on the style you want.

there's also the unfortunate side-effect that it encourages us to send
fewer, larger patches, because we keep plowing on locally. so far i've
avoided making any local commits because i don't want to find
ourselves accidentally forked, but maybe that's easier for you too? to
have a quarterly "Android sync week" or something, rather than a small
patch each week? plus that way we'll have been testing the changes for
a while.

(speaking of which, even if you don't commit it, please comment on the
TOYBOX_SELINUX patch so i know whether to go that direction for the
future SELinux tools. otherwise you're just making more work for
future you when you get the patches :-) )

we need to free you up to write awk :-)

 1421258837.0