[Toybox] Implementing ldd.

[Rob Landley]

On 1/24/22 4:52 PM, enh wrote:
>> That said, "here are the libraries" and "you have android API version skew"
>> still seem like different questions. "What libraries does this binary ask for"
>> and "does this binary fit into its filesystem" are different questions.
> 
> (exactly. lld is the latter. "recursive DT_NEEDEDs" is the former.
> please don't pollute the existing name :-) )

And now I learn why _not_ to get my bikeshedding in up front. (Easier to get
forgiveness than permission.)

Sigh. You did ask nicely. And I'm not trying to make more work for you...

(Personally, I don't write code on the assumption everybody else is dumber than
me and that I know what people I've never met want/need better than they do.)

>> > no, because like i keep saying --- ldd doesn't really make sense as a
>> > host tool :-)
>>
>> The uclibc guys thought it did.
> 
> (other than you mentioning them as your usual example of broken stuff,
> i've literally never heard of them.
It was the main alternative to glibc for 10 years. (Roughly 2003 to 2013.) My
Aboriginal Linux project was build around uClibc, there was a uClibc version of
Debian, Gentoo Embedded was uClibc based... It was a really big deal back in the
day.

The same maintainer (Erik Andersen) ran Busybox, uClibc, and buildroot. The
backstory is that back in the 1990s Caldera Embedded spun out a company called
Lineo, which had a project called uClinux to port Linux to nommu hardware. It
was run by Jeff Dionne, who tasked Erik with getting a libc and userpace that
worked on nommu systems. (Jeff reported to Lineo's CTO, Tim Bird.)

Alas Lineo went under in 2002 (shortly after the dot-com crash), and while Erik
continued busybox and uclibc in his spare time he got overwhelmed and burned out.

I learned all this stuff after the fact, of course: when I took over busybox
from Erik in 2005 I didn't know its history beyond "thing Erik does". I met Tim
Bird in 2006 (working at Sony) and Jeff Dionne in 2014 (hired me to work on
SEI/J-core; he moved to Japan when the dot-com crash happened and went back to
the hardware world). I still haven't met Erik in person but we spoke on the
phone to set up those stupid GPL enforcement suits. (Alas, hindsight...)

I already wrote this history up semi-properly once upon a time:

https://landley.net/aboriginal/history.html

The biggest single problem was that the uClibc test harness grew legs and turned
into a major build system (called "buildroot"):
http://lists.busybox.net/pipermail/uclibc/2003-August/027531.html

Which came to totally dominate traffic on the uclibc mailing list so nobody was
talking about the library's development anymore. Here's where I created a new
buildroot mailing list to kick buildroot off the uClibc mailing list so it would
stop completely drowning out the traffic there:
https://lists.buildroot.org/pipermail/buildroot/2006-July/012219.html

Here's the cake I had sent Erik to taunt him into cutting a uClibc release. It
was going to be a birthday cake for the 1 year anniversary of the previous
release, but he knew it was coming and worked to make me change the inscription:
http://landley.net/pictures/uclibc-cake-2005-1.jpg

Here's where I staged a coup and replaced the burned-out uClibc maintainer
(well, replaced the successor he'd appointed who vanished... don't get me
started on Mike and the proprietary Blackfin fork):
http://lists.busybox.net/pipermail/uclibc/2008-October/041148.html

Alas Bernhard only lasted a year or two before burning out himself. Back during
the "squashed by buildroot" phase, three different NPTL implementations had been
submitted (for three different architectures), and the devs kept trying to unify
instead of just PICKING one and throwing the rest away. This led to a two year
gap with no releases (see https://uclibc.org/oldnews.html between 0.9.28 and
0.9.28.1) where code kept piling up in the dev branch diverging farther and
farther from the release branch. I had my hands full with busybox so couldn't
take it on, and then the way I left busybox didn't leave me inclined to go back
for seconds. (Plus, Bernhard was nominally doing the thing, except for the part
where he wasn't. I note that when I burned out on busybox because of Bruce's
GPLv3 rants, Erik advised me to take a few months off instead of handing off the
project to somebody who still had enthusiasm for it. I didn't listen.)

The decline of uClibc led to eglibc (http://www.eglibc.org/home) and to Rich
making the first public release of musl. When I first bumped into Rich back on
the busybox list, his library was a private thing for his own use, ala
http://lists.busybox.net/pipermail/busybox/2006-September/058255.html

Oh, P.S. here's me shaking the tree and somebody stepping up to maintain
buildroot after Erik basically vanished (sold his startup and went off to spend
time with his family and repair his marriage):
http://lists.busybox.net/pipermail/busybox/2008-November/067751.html

The server had already moved from a machine in Erik's basement to
https://osuosl.org/ hosting, bandwidth got a bit much for his DSL line. :)

> i'm guessing they're an obsolete
> ancestor of newlib/picolibc? not the best recommendation :-) )

Nah, newlib was a Cygnus project, Red Hat aquired them using their IPO money
back around 2000.

https://en.wikipedia.org/wiki/Red_Hat#Acquisitions

Newlib was never particularly interesting, it's basically the minimum amount of
libc stub to let gcc build libg++ and then once the compiler's built you throw
it away. It's a toy project like klibc (which was more or less replaced by
https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h
these days).

>> You're coming up with questions the tool can't answer and using that to argue
>> against toybox having the tool. Many of these are questions existing version of
>> the tool can't answer either.
> 
> (no, not really ... i'm saying something more like "look, real people
> are already confused enough, and i think this idea would *increase*
> rather than *decrease* confusion". like we agreed at the top,

"You seem to be objecting to" counts as agreement?

Sigh. You did ask nicely...

> i think
> as long as this is called something that makes it clear what it
> *actually* is, it's not completely useless. though the few people who
> genuinely *only* need "recursive DT_NEEDEDs" and know that are also
> fully capable of writing the script themselves if/when they need it, i
> think. as always, i'm not too worried about them. i'm worried about
> the folks who have no idea what they're doing, and how we make sure
> they don't get even more misled.)

"Unix gives you enough rope to shoot yourself in the foot." - Allen Holub

What people who have no idea what they're doing need is a tutorial.

>> Not really seeing that as an improvement.
> 
> that's because you disagree with the three main (only?)

surviving

> ldds about what ldd does :-)

Yes, so did the man page, that's why I quoted it. The BSD ldd predates the linux
ones and is a bit more explicit:

https://www.freebsd.org/cgi/man.cgi?ldd

And the a.out ldd definitely worked like I'm thinking, before Linux switched to
ELF in 1995. :)

*shrug* I still think the older versions had it right, or at least were doing
something useful.

>> I may not be a good baseline for "they'll never need that". I've stuck printfs
>> into more than one board's stage 1 bootloader (spinning on "output byte to
>> register" and "wait for bit to clear so I can send next byte"). I stuck printfs
>> into the uClibc dynamic loader to debug its relocation of _itself_. I've worked
>> out the constant to subtract from "string" to compensate for the "copy from
>> flash to sram" it hasn't done yet but the symbol tables think it has.
> 
> (see my earlier comment about the value to people who know what
> they're doing < the added confusion for those who don't.)

I didn't expect you to enable it in the android build any time soon. :)

>> You seem to be thinking of something that almost needs to solve the halting
>> problem to figure out what new way du jour android package authors manage to
>> screw up their builds.
> 
> no, i'm saying "80% of people who ask me for 'an ldd i can run on the
> host' are *actually* looking for that verification tool instead".

I think "people asking you" is at _least_ as skewed from the norm as my "I may
not be a good baseline for" above. :)

> just you talking about it is giving me nightmares[1]. i already have a
> hard enough time trying to explain to these people that (a) that's

I didn't know "this is the list of libraries this binary is trying to load" was
a nightmarish question for you.

> basically a shell one-liner but (b) it wouldn't even find the specific
> problem with their apk that made them claim to need it, let alone any
> of the other common ones. (the worst part of all this being that i

Obviously a host tool is not going to notice selinux rule conflicts, but I have
a personal history with "or we'd have written it already" being annoyingly wrong.

> genuinely don't think there's a decent *host* implementation of this,
> or we'd have written it already.

I have been trying to get people to steal my ideas for decades, and yet my 2005
initramfs document said it worked with tmpfs which wasn't TRUE until I submitted
patches to make it so eight years later. And I still need to resubmit the darn
patch to make CONFIG_DEVTMPFS_MOUNT work on initramfs for a FIFTH time (the
third submission was in 2017 and it's 2022).

Good grief, this list was 12 years ago now and there are still multiple pending
todo items on it:

https://lists.linuxfoundation.org/pipermail/celinux-dev/2010-January/000292.html

Quick story: before uClibc created buildroot then they had a gcc wrapper that
rewrote the command line arguments to link against uClibc instead of glibc, and
then a couple years later the gcc devs invented libgcc.so, which dynamically
links against the libc gcc was compiled against. So if you did things like "64
bit division on a 32 bit processor" it would suck in libgcc.so, which sucked in
libc.so which was glibc not uClibc, and boom you're linked against two libcs and
bad things happen (such as crosslinking two independently tracked heaps).

The uClibc developers' response was to write a makefile that built a
gcc+binutils toolchain from source, then extended the makefile to smoketest the
new toolchain by building busybox and User Mode Linux to run it under (basically
to chroot as a normal user), and THEN every time they got a package completely
building under uClibc (which in the early days required patching the packages or
./configuring them strangely) they added that package to buildroot as a
regression test, and people started submitted more packages they found useful to
have, and it turned into a distro builder with hundreds of packages and sucked
up all Erik's time. (Implemented as MAKEFILES, which is a crazy way to do it and
was always my primary objection.)

My response was to fix the wrapper script to always use libgcc.a instead of
libgcc.s, even when otherwise dynamically linking. They said it couldn't be done
but I built Linux From Scratch and half of BLFS under the result just fine. When
I did get around to building portable toolchains, they used that updated build
wrapper that rewrote gcc's command line, and lots of people used my toolchains
to build all sorts of stuff. Apparently including rootkits:

https://threatpost.com/mirai-variant-cross-compiles-attack-code-with-aboriginal-linux/136906/

So yeah, I have personal experience that creating generic infrastructure can
have unintended consequences, which doesn't mean I've stopped doing it.

But you did ask nicely and I don't want to give you nightmares...

> "you need to test on the OS releases
> you claim to support" is annoying,

Running on current android doesn't prove it runs on older versions, therefore
the host should not have ldd?

This tool answers the question the man page says it answers, not the question
you think you're asking, therefore it is a bad tool?

No static analysis is going to prove it RUNS on the target. That's literally the
halting problem. In theory this is what QEMU and parallel VMs are for...

> sure, but i'm pretty sure it's a
> fact of life.)

I'm tempted to tell you to forward them to me for a bit and I'll see if I can
collect actual requirements, but I'm currently supposed to be learning the
difference between the OSU180 and SKY130 cell libraries' effect on routing the
instruction decoder, and then help figure out if the version of OSU180 ported TO
SKY130 might have fewer design rule check failures than the production cells...

(I'm pretty much Igor holding the test tubes on this one, but trying to learn,
and that's not leaving a lot of spare bandwidth just now...)

> ____
> 1. although one possibility is that they stop bothering us and start
> bothering you instead, sadly i doubt that and suspect that i spend my
> time explaining why we won't be adding "toybox ldd" to the NDK...

The current ldd says "not a dynamic executable" when confronted with something
like system/chre/build/app_support/google_slpi/libchre_slpi_skel.so but COULD
instead say "arch mismatch (use ldd --force)", or "/lib/ld-musl-powerpc.so.1 not
found (use --force)" and then output more info in --force mode. If both
objections are to stderr, it's technically compliant. :)

But if the thought gives you nightmares...

Rob