[Toybox] Is binutils neccessary?

[Rob Landley]

On 12/11/14 12:58, stephen Turner wrote:
> Rob and list,
> 
> I see there are plans to add "ar" to toybox.

Pretty much just because busybox had one. Otherwise I'd have thrown it
in the qcc todo list nstead.

(The "make" command is another corner case. It could go in toybox, it
could go in qcc. I need to write one eventually, though. All the people
trying to replace toolchains never seem to think of _all_ the stuff they
need. I did aboriginal linux to figure _out_ what the minimal set was...)

> this would be compatible
> with binutils correct?

The ar format is trivial. It's an archiver without subdirectories, it
bundles together a list of files into a single file, but the bundle has
a flat namespace.

> I think musl-libc has a built in linker
> compatible with "ld" in binutils by creating a symlink to libc.so so
> then we would only need "as" to have a fully functioning compile suite
> correct? I am assuming the presence of a working compiler.

Um, no. I have notes on this for qcc:

http://landley.net/hg/qcc/file/tip/todo/todo.txt

The linker needs support for linker scripts, things like the kernel
provide really extensive ones to micromanage elf section layout and so
on. In order to build musl the linker needs -bsymbolic-functions. and
it's really really nice if the linker can do the garbage collection
trick discarding unreferenced sections (-ffunction-sections
-fdata-sections and then -Wl,--gc-sections). Plus there's the
--as-needed stuff, and --start-group --end-group.

Don't get me started on library search paths and rpath. (Mostly the
compiler tells the linker what paths to use, but the linker then has to
go do it.)

Oh, and there's static linking, creating .o files vs exe vs .so files, a
couple flavors of PIC vs non-pic (which is oddly enough orthogonal to
static vs dynamic linking). And then that whole hash table nonsense for
performing faster lookups. Weak symbols...

The guy who wrote the "gold" linker did a marvelous blog series on what
was involved. Somebody indexed it here:

https://lwn.net/Articles/276782/

There's also a "linkers and loaders" book that is INSANELY dry, but goes
over the basics pretty well:

http://www.iecc.com/linker/

The actual elf specs are on refspecs.linuxfoundation.org (the system v
4.1 base spec plus a processor specific version is enough for most
purposes; there are some extensions though like that hash table thing
and what the compiler has to add to support for lazy binding, I'm not
sure where those are documented. Years ago I read the kernel's
fs/binfmt_elf.c (the actual kernel elf loader source) and then read
uClibc's library loader. That doesn't cover _creating_ them, just
consuming them, and musl's already got consuming them covered.)

I note that tinycc has a reasonable start to a linker (it works, with
basic linker script support), which I mean to rip out into its own .c
file and clean up as part of qcc.

A linker isn't enough. An assembler isn't enough. The linux kernel won't
_compile_ without nm. (It builds a .o file then parses the nm output
with sed to determine structure offsets to generate a header file used
by the startup code, or something like that. Run
"grep -rw NM arch" in the kernel source and look at the output.)

I found this out in aboriginal: if I'm setting $PATH to a directory I
populate in the "airlock step" (host-tools.sh, I did a presentation on
all this in 2008, linked from http://landley.net/aboriginal), then I
need to symlink "ar as nm cc gcc make ld" out of the host $PATH in
addition to building busybox and toybox. It needs _all_ of that to
bootstrap a system to a shell prompt with native toolchain.

Oh, and I vaguely recall that something was doing a disassembly with
objdump -d and running grep against the result. I don't remember why.
Not in the seven base packages of aboriginal though, I think that was in
Linux From Scratch somewhere, or maybe something depending on x11...?

> I see a number of programs in the binutils suite and am just curious of
> which are required now that i have started to find built in alternatives
> like the libc built in ld and the future toybox ar etc. 

The todo.txt link above goes through most of them, here's another file
where I just did a list of the commands binutils produces and tried to
figure out why:

http://landley.net/hg/qcc/file/tip/todo/commands.txt

There's a reason I haven't reopened the qcc can of worms before toybox
goes 1.0. It's one of them "will eat your brain" things, I'm not really
smart enough to easily wrap my head around it when that's _all_ I need
to keep track of. Doing it as a sideline to a sideline ($DAYBJOB, then
toybox, then qcc) I'd forget stuff faster than I learned it.

Rob

 1418407195.0