[Toybox] gcc vs llvm -static-libasan

[Rob Landley]

On 7/9/24 09:02, enh wrote:
> On Tue, Jul 9, 2024 at 8:47 AM Rob Landley <rob at landley.net> wrote:
>> Any suggestions?
> 
> none you're going to like, but...

I don't have to like it, I just have to make it work. :)

> you're basically checking all the "not supported" boxes here:

Welcome to my life.

> x86-64 (as opposed to arm64),

The android-ndk-r26d/toolchains/llvm/prebuild directory only has linux-x86_64
binaries. (The download page hass Platform "Linux 64-bit (x86)" as the only
Linux option. It can produce arm output, but doesn't provide a toolchain that
would run on an arm system, therefore a native compile-and-test cycle...

> asan (as opposed to hwasan),

Which wasn't available on x86-64 last I checked.

I could whip up a raspberry-pi alike if pushed, but if you don't publish an ndk
that runs there... (I could try building it from source again?)

Adding qemu system emulation into the compile and test cycle is on the todo
list. Adding qemu APPLICATION emulation is a recipe for weird corner cases in
system call translation and something I've generally tried to avoid because
system call translation is a categorically harder problem than emulating
hardware interfaces. (Containers leverage the existing kernel, based on work
done by OpenVZ since 1999, running the same host/guest binaries with all the
endianness and page size and magic constants and structure packing issues
guaranteed identical, and it STILL took 10 to get it load bearing.)

> static (as opposed to dynamic).

Again, running on a debian host. When they come out with debian-bionic I'll
happily debootstrap a chroot for testing.

Getting an android chroot to work as a development environment is the goal,
hence chicken-and-egg problem.

> i believe that arm64 hwasan static binaries "just work",
> and if they don't, that's something we'd actually be able to find the
> time to look at. until there's x86-64 hardware with whatever they're
> calling "x86-64 top byte ignore", though, you're stuck with asan, and
> that's been known to be mostly/somewhat broken on Android for years
> (with no-one to work on it).
> 
> why can't you run the dynamic version? i thought you'd upgraded your
> laptop?

I did a fresh OS install on a spare laptop of the same model, but it's still a
glibc system.

> is the new one also too old to run ~2012-era x86-64 binaries?

It runs the static binaries just fine, but it's a glibc system: "./toybox:
cannot execute: required file not found". (And then "ldd toybox" said
"/usr/lib/x86_64-linux-gnu/libm.so: invalid ELF header" which isn't _entirely_
surprising given that gnu/ldd is a bash script calling glibc but was another
"wha...?" moment I did not pursue further.)

I vaguely remembered you saying you could create a "/system" symlink into the
ndk somewhere, so I ran readelf on the binary which is requesting
/system/bin/linker64 and did a "find android-ndk-r26d -name linker64" and there
were no hits, so even setting up a dynamic chroot for it remains a todo list item.

> alternatively, build arm64 hwasan binaries and run them on your phone
> or orangepi?
> 
> i'll forward this to the llvm asan folks anyway, in particular the
> "gcc and llvm use different flags" is the kind of papercut that i
> think both gcc and llvm have been trying harder to avoid lately.

Thanks.

As long as ASAN works for me _somewhere_ I don't really care where (should be
the same bugs), and I mostly want all the tests to pass with ASAN before cutting
a release (which is due again: time flies like an arrow, fruit flies like an
apple, of the two time flies eating all your arrows sounds like the bigger
problem...).

But if I can't check in the "make it work with gcc on debian" flag without
breaking the Android build, I has a sad. :(

Rob