[Toybox] Implementing ldd.

enh enh at google.com
Fri Jan 28 18:09:59 PST 2022

On Tue, Jan 25, 2022 at 12:59 PM Rob Landley <rob at landley.net> wrote:
> On 1/24/22 4:52 PM, enh wrote:
> >> That said, "here are the libraries" and "you have android API version skew"
> >> still seem like different questions. "What libraries does this binary ask for"
> >> and "does this binary fit into its filesystem" are different questions.
> >
> > (exactly. lld is the latter. "recursive DT_NEEDEDs" is the former.
> > please don't pollute the existing name :-) )
> And now I learn why _not_ to get my bikeshedding in up front. (Easier to get
> forgiveness than permission.)
> Sigh. You did ask nicely. And I'm not trying to make more work for you...
> (Personally, I don't write code on the assumption everybody else is dumber than
> me and that I know what people I've never met want/need better than they do.)

whereas i have whole weeks where i'm basically just very expensive
tech support for random folks off the internet :-)

fwiw, i did actually check the bug database for requests along these
lines (for data versus the "anecdata" you've already heard), and all i
found were people who really did want to know *which exact* libfoo.so
is going to be loaded.

> >> > no, because like i keep saying --- ldd doesn't really make sense as a
> >> > host tool :-)
> >>
> >> The uclibc guys thought it did.
> >
> > (other than you mentioning them as your usual example of broken stuff,
> > i've literally never heard of them.
> It was the main alternative to glibc for 10 years. (Roughly 2003 to 2013.) My
> Aboriginal Linux project was build around uClibc, there was a uClibc version of
> Debian, Gentoo Embedded was uClibc based... It was a really big deal back in the
> day.
> The same maintainer (Erik Andersen) ran Busybox, uClibc, and buildroot. The
> backstory is that back in the 1990s Caldera Embedded spun out a company called
> Lineo, which had a project called uClinux to port Linux to nommu hardware. It
> was run by Jeff Dionne, who tasked Erik with getting a libc and userpace that
> worked on nommu systems. (Jeff reported to Lineo's CTO, Tim Bird.)
> Alas Lineo went under in 2002 (shortly after the dot-com crash), and while Erik
> continued busybox and uclibc in his spare time he got overwhelmed and burned out.
> I learned all this stuff after the fact, of course: when I took over busybox
> from Erik in 2005 I didn't know its history beyond "thing Erik does". I met Tim
> Bird in 2006 (working at Sony) and Jeff Dionne in 2014 (hired me to work on
> SEI/J-core; he moved to Japan when the dot-com crash happened and went back to
> the hardware world). I still haven't met Erik in person but we spoke on the
> phone to set up those stupid GPL enforcement suits. (Alas, hindsight...)
> I already wrote this history up semi-properly once upon a time:
> https://landley.net/aboriginal/history.html
> The biggest single problem was that the uClibc test harness grew legs and turned
> into a major build system (called "buildroot"):
> http://lists.busybox.net/pipermail/uclibc/2003-August/027531.html
> Which came to totally dominate traffic on the uclibc mailing list so nobody was
> talking about the library's development anymore. Here's where I created a new
> buildroot mailing list to kick buildroot off the uClibc mailing list so it would
> stop completely drowning out the traffic there:
> https://lists.buildroot.org/pipermail/buildroot/2006-July/012219.html
> Here's the cake I had sent Erik to taunt him into cutting a uClibc release. It
> was going to be a birthday cake for the 1 year anniversary of the previous
> release, but he knew it was coming and worked to make me change the inscription:
> http://landley.net/pictures/uclibc-cake-2005-1.jpg
> Here's where I staged a coup and replaced the burned-out uClibc maintainer
> (well, replaced the successor he'd appointed who vanished... don't get me
> started on Mike and the proprietary Blackfin fork):
> http://lists.busybox.net/pipermail/uclibc/2008-October/041148.html
> Alas Bernhard only lasted a year or two before burning out himself. Back during
> the "squashed by buildroot" phase, three different NPTL implementations had been
> submitted (for three different architectures), and the devs kept trying to unify
> instead of just PICKING one and throwing the rest away. This led to a two year
> gap with no releases (see https://uclibc.org/oldnews.html between 0.9.28 and
> where code kept piling up in the dev branch diverging farther and
> farther from the release branch. I had my hands full with busybox so couldn't
> take it on, and then the way I left busybox didn't leave me inclined to go back
> for seconds. (Plus, Bernhard was nominally doing the thing, except for the part
> where he wasn't. I note that when I burned out on busybox because of Bruce's
> GPLv3 rants, Erik advised me to take a few months off instead of handing off the
> project to somebody who still had enthusiasm for it. I didn't listen.)
> The decline of uClibc

(i actually asked the internet after hitting send, and was amused by
the wikipedia page's "final release: 2012", specifically how far past
your usual 7 year rule that was :-) )

> led to eglibc (http://www.eglibc.org/home) and to Rich
> making the first public release of musl. When I first bumped into Rich back on
> the busybox list, his library was a private thing for his own use, ala
> http://lists.busybox.net/pipermail/busybox/2006-September/058255.html
> Oh, P.S. here's me shaking the tree and somebody stepping up to maintain
> buildroot after Erik basically vanished (sold his startup and went off to spend
> time with his family and repair his marriage):
> http://lists.busybox.net/pipermail/busybox/2008-November/067751.html
> The server had already moved from a machine in Erik's basement to
> https://osuosl.org/ hosting, bandwidth got a bit much for his DSL line. :)
> > i'm guessing they're an obsolete
> > ancestor of newlib/picolibc? not the best recommendation :-) )
> Nah, newlib was a Cygnus project, Red Hat aquired them using their IPO money
> back around 2000.
> https://en.wikipedia.org/wiki/Red_Hat#Acquisitions
> Newlib was never particularly interesting, it's basically the minimum amount of
> libc stub to let gcc build libg++ and then once the compiler's built you throw
> it away. It's a toy project like klibc (which was more or less replaced by
> https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h
> these days).
> >> You're coming up with questions the tool can't answer and using that to argue
> >> against toybox having the tool. Many of these are questions existing version of
> >> the tool can't answer either.
> >
> > (no, not really ... i'm saying something more like "look, real people
> > are already confused enough, and i think this idea would *increase*
> > rather than *decrease* confusion". like we agreed at the top,
> "You seem to be objecting to" counts as agreement?
> Sigh. You did ask nicely...
> > i think
> > as long as this is called something that makes it clear what it
> > *actually* is, it's not completely useless. though the few people who
> > genuinely *only* need "recursive DT_NEEDEDs" and know that are also
> > fully capable of writing the script themselves if/when they need it, i
> > think. as always, i'm not too worried about them. i'm worried about
> > the folks who have no idea what they're doing, and how we make sure
> > they don't get even more misled.)
> "Unix gives you enough rope to shoot yourself in the foot." - Allen Holub
> What people who have no idea what they're doing need is a tutorial.

tutorials don't scale, and bitrot really fast. (i literally spent this
afternoon trying *and failing* to follow our "how to use asan for an
app" docs.) what people really need is automation, and the great thing
about automation is that it's good for the experts as well as the

> >> Not really seeing that as an improvement.
> >
> > that's because you disagree with the three main (only?)
> surviving
> > ldds about what ldd does :-)
> Yes, so did the man page, that's why I quoted it. The BSD ldd predates the linux
> ones and is a bit more explicit:
> https://www.freebsd.org/cgi/man.cgi?ldd
> And the a.out ldd definitely worked like I'm thinking, before Linux switched to
> ELF in 1995. :)

(TIL a.out supported shared libraries! that must have been while i was
busy statically linking everything/only writing java anyway...)

> *shrug* I still think the older versions had it right, or at least were doing
> something useful.
> >> I may not be a good baseline for "they'll never need that". I've stuck printfs
> >> into more than one board's stage 1 bootloader (spinning on "output byte to
> >> register" and "wait for bit to clear so I can send next byte"). I stuck printfs
> >> into the uClibc dynamic loader to debug its relocation of _itself_. I've worked
> >> out the constant to subtract from "string" to compensate for the "copy from
> >> flash to sram" it hasn't done yet but the symbol tables think it has.
> >
> > (see my earlier comment about the value to people who know what
> > they're doing < the added confusion for those who don't.)
> I didn't expect you to enable it in the android build any time soon. :)

yeah, in that sense i don't care: i won't build for android or for the
host. but that doesn't mean i won't get asked for it by people who
have no idea what they're even saying (or at least can't word it in a
way i can interpret). http://b/215403271 "add busybox for android".

> >> You seem to be thinking of something that almost needs to solve the halting
> >> problem to figure out what new way du jour android package authors manage to
> >> screw up their builds.
> >
> > no, i'm saying "80% of people who ask me for 'an ldd i can run on the
> > host' are *actually* looking for that verification tool instead".
> I think "people asking you" is at _least_ as skewed from the norm as my "I may
> not be a good baseline for" above. :)
> > just you talking about it is giving me nightmares[1]. i already have a
> > hard enough time trying to explain to these people that (a) that's
> I didn't know "this is the list of libraries this binary is trying to load" was
> a nightmarish question for you.

see the bug i just linked to for an example of the quality of bug
report i deal with :-)

> > basically a shell one-liner but (b) it wouldn't even find the specific
> > problem with their apk that made them claim to need it, let alone any
> > of the other common ones. (the worst part of all this being that i
> Obviously a host tool is not going to notice selinux rule conflicts, but I have
> a personal history with "or we'd have written it already" being annoyingly wrong.

my point is more "it _has_ been written already, and came with your libc".

i think your point is "i want something for the host when
cross-compiling for embedded systems (and am happy to ignore even
stuff like rpath or $LD_LIBRARY_PATH)", to which my only response is
"i don't get why you don't write the short script like everyone else,
but if you prefer C, that's fine, but please don't increase confusion
by calling it ldd".

> > genuinely don't think there's a decent *host* implementation of this,
> > or we'd have written it already.
> I have been trying to get people to steal my ideas for decades, and yet my 2005
> initramfs document said it worked with tmpfs which wasn't TRUE until I submitted
> patches to make it so eight years later. And I still need to resubmit the darn
> patch to make CONFIG_DEVTMPFS_MOUNT work on initramfs for a FIFTH time (the
> third submission was in 2017 and it's 2022).
> Good grief, this list was 12 years ago now and there are still multiple pending
> todo items on it:
> https://lists.linuxfoundation.org/pipermail/celinux-dev/2010-January/000292.html
> Quick story: before uClibc created buildroot then they had a gcc wrapper that
> rewrote the command line arguments to link against uClibc instead of glibc, and
> then a couple years later the gcc devs invented libgcc.so, which dynamically
> links against the libc gcc was compiled against. So if you did things like "64
> bit division on a 32 bit processor" it would suck in libgcc.so, which sucked in
> libc.so which was glibc not uClibc, and boom you're linked against two libcs and
> bad things happen (such as crosslinking two independently tracked heaps).
> The uClibc developers' response was to write a makefile that built a
> gcc+binutils toolchain from source, then extended the makefile to smoketest the
> new toolchain by building busybox and User Mode Linux to run it under (basically
> to chroot as a normal user), and THEN every time they got a package completely
> building under uClibc (which in the early days required patching the packages or
> ./configuring them strangely) they added that package to buildroot as a
> regression test, and people started submitted more packages they found useful to
> have, and it turned into a distro builder with hundreds of packages and sucked
> up all Erik's time. (Implemented as MAKEFILES, which is a crazy way to do it and
> was always my primary objection.)
> My response was to fix the wrapper script to always use libgcc.a instead of
> libgcc.s, even when otherwise dynamically linking. They said it couldn't be done
> but I built Linux From Scratch and half of BLFS under the result just fine. When
> I did get around to building portable toolchains, they used that updated build
> wrapper that rewrote gcc's command line, and lots of people used my toolchains
> to build all sorts of stuff. Apparently including rootkits:
> https://threatpost.com/mirai-variant-cross-compiles-attack-code-with-aboriginal-linux/136906/
> So yeah, I have personal experience that creating generic infrastructure can
> have unintended consequences, which doesn't mean I've stopped doing it.
> But you did ask nicely and I don't want to give you nightmares...
> > "you need to test on the OS releases
> > you claim to support" is annoying,
> Running on current android doesn't prove it runs on older versions, therefore
> the host should not have ldd?

more like "for my users, the recursive DT_NEEDEDs tells them *nothing*
... but they think it does, and i have enough headaches already" :-)

> This tool answers the question the man page says it answers, not the question
> you think you're asking, therefore it is a bad tool?
> No static analysis is going to prove it RUNS on the target. That's literally the
> halting problem. In theory this is what QEMU and parallel VMs are for...
> > sure, but i'm pretty sure it's a
> > fact of life.)
> I'm tempted to tell you to forward them to me for a bit and I'll see if I can
> collect actual requirements, but I'm currently supposed to be learning the
> difference between the OSU180 and SKY130 cell libraries' effect on routing the
> instruction decoder, and then help figure out if the version of OSU180 ported TO
> SKY130 might have fewer design rule check failures than the production cells...
> (I'm pretty much Igor holding the test tubes on this one, but trying to learn,
> and that's not leaving a lot of spare bandwidth just now...)
> > ____
> > 1. although one possibility is that they stop bothering us and start
> > bothering you instead, sadly i doubt that and suspect that i spend my
> > time explaining why we won't be adding "toybox ldd" to the NDK...
> The current ldd says "not a dynamic executable" when confronted with something
> like system/chre/build/app_support/google_slpi/libchre_slpi_skel.so but COULD
> instead say "arch mismatch (use ldd --force)", or "/lib/ld-musl-powerpc.so.1 not
> found (use --force)" and then output more info in --force mode. If both
> objections are to stderr, it's technically compliant. :)

bionic's linker does say "is for EM_??? (20) instead of EM_AARCH64" :-)

(i couldn't bring myself to include and maintain a set of strings for
the architectures we *don't* support!)

> But if the thought gives you nightmares...
> Rob

More information about the Toybox mailing list