[Toybox] Implementing ldd.

[enh]

(sorry for the delay; specifically because there's a ton of weird
stuff behind the scenes, it took me a while to find the time to write
a reply. and even this is just an overview!)

On Fri, Jan 14, 2022 at 9:07 PM Rob Landley <rob at landley.net> wrote:
>
> On 1/13/22 1:20 PM, enh wrote:
> > TL;DR: unless you're writing your own dynamic linker, i'm not sure why
> > you'd write an ldd?
>
> Well to start with uClibc had a standalone one back in the day:
>
>   https://git.busybox.net/uClibc/tree/utils/ldd.c
>
> Which is basically a subset of readelf, and we have readelf...
>
> The one built into glibc can't tell you anything about a musl or bionic binary
> let alone something cross-compiled for another architecture.

yeah, "i want to check my Android libraries on the host" is the use
case we always hear. unfortunately, "i think you'll find it's a bit
more complicated than that"...

in particular, these are some of the most common Android issues (none
of which even the *real* ldd handles --- they ask for "ldd on the
host", but that's never what they actually mean):
* did you copy all your libraries into your apk?
* did you put all your libraries in the right *directory* in the apk?
* did you make sure they're the right *architecture* (and that they're
in the right subdirectory for that architecture)?
* are the SONAMEs correct? (in particular: not Windows \\ paths (!))
* have you tested on older Android versions where resolution differs?
* okay, so they're the right arch and in the right directory, and the
SONAME is fine --- but you do know you can't just copy a glibc binary
from your host and drop it in your apk, right?

> I can never remember if it's "readelf -a" or "readelf -A" to fetch it with the
> more complex tool, and it's way harder to script something like
> https://git.busybox.net/busybox/tree/testsuite/testing.sh#n111 with readelf's
> output.
>
> "What libraries does this command fail if it can't load" seems a fairly
> straightforward question to ask? Usually I'm trying to figure out what
> dependencies to transplant when ripping binaries out of a system. (Disrto skew,
> version skew, hacking together an initramfs on a USB stick...)

aye, but that's just `readelf -dW | grep NEEDED`. and the nice thing
about that is that its limitations are nice and clear. ldd is a bit
magic, and your "readelf ldd" would have none of that magic. (let
alone cover the stuff that even ldd doesn't.)

[funnily enough, i see man7's ldd page basically suggests the same if
you want to look at an untrusted binary, but uses `objdump -p | grep
NEEDED` instead.]

> > personally i don't think a readelf-based ldd makes much sense? there
> > are a ton of weird special cases that mean you really do need the
> > "suck it and see" implementation
>
> Such as?

a few examples of stuff [that happens all the time] that ldd *does*
catch that "readelf ldd" wouldn't:
* symbols that don't resolve.
* DT_NEEDEDs on libraries that you're not allowed to access (a
run-time decision based on linker namespaces on Android).
* incorrect ELF notes for stuff like PAC/BTI.
* what library your DT_NEEDED *actually* resolves to (which even on
glibc involves parsing text files, no? and i don't know how glibc's
new per-arch variant optimized builds stuff works, but that's probably
something ldd does know).

> > --- bionic already just has a
> > one-line shell script that passes an argument to the dynamic linker
> > (and i think glibc and musl work this way too).
>
> Musl points an "ldd" symlink at the dynamic linker which notices the argv[0]
> name and behaves differently. (I may have made puppy eyes at Rich years ago, I'd
> have to check my back email...)

yeah, that's exactly what we do too :-)

tbh, i think it's the only option that makes sense: "hey, linker, do
you stuff, just don't actually run the code when you're finished".

> The script you mention does not appear to be shipped in the NDK:
>
>   $ find android-ndk-r24-beta1 -name '*ldd*'
>   $

no, because like i keep saying --- ldd doesn't really make sense as a
host tool :-)

check out /system/bin/ldd instead.

(or keep grepping readelf/objdump if *that's* all you want instead!)

> Or aosp prebuilts:
>
>   $ find aosp -name '*ldd'
>   aosp/external/ltp/testcases/commands/ldd
>   $
>
> But then it doesn't seem to be installed in the cross compilers musl-cross-make
> builds either. That said, I can symlink it myself:
>
> $ cd x86_64-linux-musl-cross
> $ ln -s x86_64-linux-musl/lib/libc.so ldd
>
> If I point yours at an sh4 binary linked against musl will it work?

no, because that's not the question ldd answers :-) see above.

> Because the
> musl one won't:
>
> $ ./ldd ../sh2eb-linux-muslfdpic-cross/sh2eb-linux-muslfdpic/lib/libc.so
> ./ldd: ../sh2eb-linux-muslfdpic-cross/sh2eb-linux-muslfdpic/lib/libc.so: Not a
> valid dynamic program
> $ echo 'main() {;}' |
> ../sh2eb-linux-muslfdpic-cross/bin/sh2eb-linux-muslfdpic-cc -x c -
> $ ./ldd a.out
> ./ldd: a.out: Not a valid dynamic program

that seems like the correct answer to the question *ldd* answers, in
the same way that Android's ldd can only tell you "will this work on
*this* device?". sure, x86-64 Android apps are a thing, but they may
well not run on your device. furthermore, because the usual app
System.loadLibrary() environment is different from the [less
restricted] adb shell environment, /system/bin/ldd doesn't really
answer the question for an *app* anyway. you can quite easily have
something that works fine from `adb shell` but not from an app. (aka
"my unit test works but my app doesn't".)

to answer the *app* question, you need your CI to actually start your
app on an actual device/emulator.

> But the old uClibc ldd from the dawn of time?
>
> $ wget -O-
> https://landley.net/aboriginal/downloads/old/binaries/1.4.0/cross-compiler-x86_64.tar.gz
> | tar xvz
> $ cross-compiler-x86_64/bin/x86_64-ldd a.out
>         libc.so => not found (0x00000000)
>         not a dynamic executable
>
> Well it's a little unhappy (doesn't know about fdpic for one thing), but it did
> tell me what shared library the file needed. Alas, it doesn't seem to recognize
> the ndk toolchain's output at all, probably sanity checking header flags and new
> ones have been added since...)
>
> > which isn't to say that the question doesn't sometimes come up of "i'd
> > like to print the transitive closure of DT_NEEDEDs *on the host*" but
> > note that that's *not* the same thing, and so even if someone does
> > write that, it probably deserves a different tool name and a big "note
> > that your run-time mileage may vary wildly".
>
> The ldd in uClibc showed the DT_NEEDED entries in an ELF file, and at the time
> that's basically what glibc's did too (just... badly). I used it for ~10 years
> and don't remember hitting one of the corner cases you're mentioning? (I'm
> guessing a C++ name demangling thing?)

yeah, the only time i use ldd on the *host* is indeed for "list the
transitive DT_NEEDEDs, please". but even there, i'm often specifically
interested in "which exact version of libfoo, and from which
directory". and only the real linker can give me the real answer.

i guess for me the question of whether your "fakedd" is ever going to
find itself on a system where you don't have the real thing? if you
have glibc or musl or bionic, you also have their dynamic linker, and
their dynamic linkers act as ldd if asked nicely. (and if you don't
have their dynamic linkers, you only have static binaries, so ldd
isn't meaningful for you anyway.)

> Admittedly, I wasn't trying to get vdso info out of it. I'll take you're word
> they've grown more bells and whistles since, and am curious what those are?

(see above, but, yes, "what [if anything] did the linker do about the
vdso" is another interesting question. though i'd hope anyone not
personally maintaining a dynamic linker never needs it for that :-) )

> Rich implemented his ldd as "make a symlink called ldd pointing at the dynamic
> linker and it notices the argv[0] name and acts like ldd":
>
>   http://git.musl-libc.org/cgit/musl/tree/ldso/dynlink.c?h=v1.2.2#n1793
>
> Even if I _didn't_ suggest it to him back in the day (don't remember), given my
> biases I just thought he was doing that old trick to reuse code. (Which
> historically busybox copied from gzip/gunzip being hardlinked and supplying -d
> to itself based on the name back in 1995, and I'm told there were unix v6
> commands doing that back in the 1970s...)

i can't speak for him, but for us it was definitely a "this is
*really* complicated, and the only realistic way to do it is either
build the linker twice, once with an exit() and once with a jump to
main(), or reuse the same binary". that wasn't a particular hard
choice :-)

> Rob