[Aboriginal] What's musl, anyway?

[Rob Landley]

On 10/05/2011 05:52 PM, maillist-aboriginal at barfooze.de wrote:
> let me answer the question in the subject first:
> musl is a new libc for linux, which imo is exceptionally well made.
> according to my own experience its smaller, faster and more correct than
> uclibc.

Huh, are you dalias under yet another name, or is this _another_ libc?

This is different from the attempts to port bionic to the vanilla
kernel, has no relation to klibc, is not related to uClibc, dietlibc,
glibc, libc5, or any of the BSD libc projects?

(What definitions of "correct" mean something other than "runs the
software that's out there", by the way?)

> here is a comparison: http://www.etalabs.net/compare_libcs.html
> (it even has stuff which uclibc lacks, such as posix_spawn, which is a
> really handy function)
> 
> musl has ARM support since the last release and i'm currently porting
> sabotage linux
> to build on that arch, using aboriginal as my build platform.
> there are a couple of issues (mostly the old binutils used by
> aboriginal, which don't
> go well together with musl's gcc wrapper, the lack of a debugger
> and the speed and memory restrictions of qemu), however i'm making
> progress...

Most of those are on my todo list to fix.  the memory restrictions
should be easier to lift once device tree support properly goes into
both kernel and qemu (bits are there now, maybe enough already, I have
to poke at it).  I've got a todo list item to add a -Bsymbolic-thingy
flag but haven't gotten time to work on that yet.

Have you tried building gdb natively under uClibc (statically linked)
and then using that binary under a musl chroot?

>>> also in my tests stuff was partially 50% faster than with -O3 alone.
>>> so it's actually a pretty neat feature.
>> Sounds like you're compiling badly written code, but you work with what
>> you've got...
>>
> not really, i did test a couple of different selfwritten arraylist
> implementations.
> using the -flto -fwhole-program flags, i could see big speed and size
> differences,
> when all involved c-files where thrown at once onto the compiler,
> as opposed to just linking object files together.

*shrug*  Ok.

>>> i was talking about gcc *3*.
>> Ah, I missed that.
>>
>> There are arguments for supporting older toolchain versions, and
>> arguments for supporting newer toolchain versions.  Mostly I just want a
>> working toolchain to bootstrap a target, and then you can natively build
>> a new toolchain under that in things like lfs-bootstrap.hdc.
>>
> i actually think a handful of patches on the kernel source would suffice
> to still build a complete linux system using gcc 3.
> given the immense memory requirements when compiling gcc4.5+it could
> even be possible that gcc 3 is the only vital option.

Except there are lots of target output types that doesn't support.

>> The interesting thing that newer gcc versions give you is support for
>> more targets.  For example, armv7 showed up in gcc 4.3, which is the big
>> incentive to support the newer one.  Support for the xylinx microblaze
>> would also be nice, since qemu has that now.  Alpha and m68k compilers
>> that don't die so often with internal compiler errors while cross
>> compiling stuff would also be cool, although I the native versions of
>> those compilers might be more stable.
>>
> indeed, the more targets, the better. however i've seen in gcc's
> changelog that
> in the last releases some old architectures have been removed...

Did I mention I think the FSF is really really bad at software engineering?

>>> the build time on my 3ghz machine is 5 minutes compared to 45 minutes
>>> for gcc4.5 (without mpc/mpfr/gmp, who consume another 5 minutes alone)
>>> only thing missing is the gnu99 inline stuff. apart from that it
>>> compiles 99% of the code out there.
>> It is indeed cool.  But if I recall it couldn't do simple dead code
>> elimination on arm, meaning busybox had a build break trying to link
>> code out of .c files it hadn't bothered to compile because it knew they
>> wouldn't be used.
>>
> yep, that's the tradeoff you have to make... features vs bloat :/

With FSF code, maybe.  In general, you can do efficient implementations
and get a good bang for the byte.  (THEY can't, but see "bad at
software", above.)

>>> never heard about tcg. i'll read up on that one.
>> There's a README in qemu's tcg subdirectory.  See also
>> http://127.0.0.1/qemu/2008-01-29.html#Feb_1,_2008_-_TCG
>>
> thanks, that looks pretty interesting.
> the level of optimization done is tiny, though (no idea how much it was
> improved since then).

But the time it takes to compile code is also tiny.  (This code
generator is on par with if not faster than the one in tcc.)

>>> i can load the file into vi, both using the symlink and the link target.
>>> musl's readdir is just a one liner around the kernel syscall.
>> Use strace to see what arguments it's passing to the syscall.
>>
> 
> yep, using strace i could hunt down the bug.
> i was previously inserting printf's into musl code and recompiling
> everytime due to a lacking debugger...
> not really used to that kind of debugging. getting a working statically
> compiled gdb for ARM is on the
> top of my TODO list.

Have you tried building it natively under either dev-environment.sh's
/home or under the filesystem lfs-bootstrap.hdc produces?

> musl had fcntl.h with values from i386, which slightly differ from ARM, so
> the O_LARGEFILE of musl's open() was interpreted as O_NOFOLLOW from the
> kernel.
> was fixed yesterday by
> http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=e6d765a8b1278e9e5f507638ccdec9fe40e52364

Yeah that'll do it.

> (that's also a nice thing about musl, i report the bug and 10 minutes
> later it is fixed, usually)

Cool.  I try to fix issues the same week they're reported (clearing my
backlog on weekends), but don't always succeed.  (I have a day job and
working on this isn't it.)

Rob

 1317938488.0