[Aboriginal] What's musl, anyway?

[John Spencer]

On 10/07/2011 05:47 AM, Rob Landley wrote:
> On 10/06/2011 06:20 PM, John Spencer wrote:
>> On 10/07/2011 12:01 AM, Rob Landley wrote:
>>> On 10/05/2011 05:52 PM, maillist-aboriginal at barfooze.de wrote:
>>>> let me answer the question in the subject first:
>>>> musl is a new libc for linux, which imo is exceptionally well made.
>>>> according to my own experience its smaller, faster and more correct than
>>>> uclibc.
>>> Huh, are you dalias under yet another name,
>> no. but dalias is indeed the author of musl libc.
>> btw, he used aboriginal to make the arm port.
> Ok, I've talked to him on and off for years in #edev and #uclibc and the
> busybox mailing list and such, I'm aware of his effort.  Sounds like
> fun, I'm just trying to keep track. :)
>

ah, i see.

>>> or is this _another_ libc?
>> it is another libc if compared to the ones mentioned below.
>> not sure what you're refering to here.
> It's nice to have a good small simple libc.  It would be _really_ nice
> to document the steps to adding a new target.

here's a writeup from the amd64 port:
http://brightrain.aerifal.cx/~niklata/PORTING

this commit here documents about everything needed for the arm port
http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=d960d4f2cbf18ff3476c7ac03698ec253885dd8e

and these two here add dynamic linking to musl arm
http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=12ace5bf76040e928df1abd57e9b5134ca4647c6
http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=fcaf7065698bce12a444803587421ae88446f6c0

see how little code was needed...
> (In aboriginal, all the information about a specific target is in the
> sources/targets files, one per target.  At one point many moons ago i
> had documentation about adding a new target.  um...
> http://landley.net/aboriginal/documentation.html#new_target I think.
> Wow that's out of date...)
>
>>> This is different from the attempts to port bionic to the vanilla
>>> kernel, has no relation to klibc, is not related to uClibc, dietlibc,
>>> glibc, libc5, or any of the BSD libc projects?
>>>
>> yep, different attempt. and so far it's looking very sweet. couldn't do
>> it better myself.
>> completeness status is reflected in the version number which is 0.8.xx
>> there's some C99 math stuff missing and no C++ support yet, but apart
>> from that nearly everything compiles now
> Why should libc care about c++ again?  C++ is not C, any more than
> Objective C is.  Java borrowed a lot of C's syntax, and isn't C either.
>

indeed, i actually would prefer a system without any C++ crap.
however there are many useful apps written in it (like gold), and to get 
wide acceptance musl has to get some way to build C++ stuff. it's 
unlikely that the GCC guys will do it...
>> (after fixing broken/idiotic build systems and gnu assumptions, of course)
>>> (What definitions of "correct" mean something other than "runs the
>>> software that's out there", by the way?)
>>>
>> one example for correctness is that musl doesnt implement a 32bit off_t
>> api, because if you allow both you can have
>> a 32-bit off_t program invoked by a 64-bit off_t shell...
>> its stdin/out/err will be in 64-bit mode...
> Removing obsolete broken crap: yay!
>
> You'd think uClibc would be the project for this, but that was before:
>
>    http://lists.busybox.net/pipermail/uclibc/2007-September/039215.html
>
> And although I spent YEARS nudging it back together (and out from under
> buildroot's shadow), it's nowhere near recovered yet and I've got too
> much else to do...
>
>>>> here is a comparison: http://www.etalabs.net/compare_libcs.html
>>>> (it even has stuff which uclibc lacks, such as posix_spawn, which is a
>>>> really handy function)
>>>>
>>>> musl has ARM support since the last release and i'm currently porting
>>>> sabotage linux
>>>> to build on that arch, using aboriginal as my build platform.
>>>> there are a couple of issues (mostly the old binutils used by
>>>> aboriginal, which don't
>>>> go well together with musl's gcc wrapper, the lack of a debugger
>>>> and the speed and memory restrictions of qemu), however i'm making
>>>> progress...
>>> Most of those are on my todo list to fix.  the memory restrictions
>>> should be easier to lift once device tree support properly goes into
>>> both kernel and qemu (bits are there now, maybe enough already, I have
>>> to poke at it).  I've got a todo list item to add a -Bsymbolic-thingy
>>> flag but haven't gotten time to work on that yet.
>> actually it would be sweet to have ld.gold around, but that'd require a
>> working C++ env
>> (at least to build it, statically preferred ;))
> In theory, the system images I built and posted have working c++
> toolchains in them, based on uClibc+=.  Haven't stressed it much, but it
> builds hello world with the funky "we redefined the bit shift operators
> to perform I/O" stuff.
>
> (It died building cmake, I have to go back and work out why...)
>

not really in the mood to fight with C++ code currently.
btw, i could compile gcc 4.5.3 by passing -Wl,--no-keep-memory to the 
linker.
i also added a patch which disables target-libiberty and target-libz.
with disable-bootstrap the compiler built in around 15 minutes using 
distcc o.0
so it actually looks as if --no-keep-memory actually is *faster* despite 
being described as "slow but less memory hungry"

>> i was told that it is faster and takes way less ram than gnu ld.
>> (which is not very surprising, given the quality of average gnu code)
> The tinycc linker is also buckets faster and less memory intensive than
> gnu ld.

how about this one ? http://code.google.com/p/ucpp/
> Too bad the maintainer of that project is a windows developer,
> if it would just DIE I'd restart my fork but I refuse to fight zombies
> without a cricket bat.

hmm iirc the mob branch is the only active. it's not even that bad, i 
checked it out lately and it built stuff which tcc 0.9.25 refused to build.
>>>> indeed, the more targets, the better. however i've seen in gcc's
>>>> changelog that
>>>> in the last releases some old architectures have been removed...
>>> Did I mention I think the FSF is really really bad at software
>>> engineering?
>>>
>> full ack. what sucks most tho are the build systems used by gnu.
>> i guess to fully grok gcc's build system you have to become Phd at the
>> gnu university.
> http://landley.net/notes-2010.html#11-11-2010
> http://landley.net/notes-2010.html#15-10-2010
> http://landley.net/notes-2010.html#26-08-2010
> http://landley.net/notes-2010.html#20-08-2010
>
> And so on, and so forth.  Trust me: they don't understand it either.
> They just keep adding another special case for each new test case.
>
> Literally.  That is the design of autoconf.
>

interesting lecture. i actually think it would make sense to writeup a 
"serious" blog post [web 2.0 like with comments and such] about 
autoconf, with some examples like these here and link it on reddit.
compiling stuff could be so much fun if autoconf'd build systems 
wouldn't always fuck everything up.
actually it is so broken that it defeats the purpose of the FSF, in that 
it makes it too hard to change and recompile your free software, making 
it unfree that way.

>>>> thanks, that looks pretty interesting.
>>>> the level of optimization done is tiny, though (no idea how much it was
>>>> improved since then).
>>> But the time it takes to compile code is also tiny.  (This code
>>> generator is on par with if not faster than the one in tcc.)
>>>
>> sweet. i imagine a compiler which could crosscompile to any wished
>> target by just adding a flag.
> Basically, yeah. :)
>
> You have to add said flag to the preprocessor as well, it sets a bunch
> of symbols telling you about your current target.  Go:
>
>    gcc -E -dM -<  /dev/null | less
>
> Now do that for each of the cross compilers and diff the result. :)

... thats quite a list.
>>> Have you tried building it natively under either dev-environment.sh's
>>> /home or under the filesystem lfs-bootstrap.hdc produces?
>>>
>> not yet, will try later today. in fact i ported gdb to build on musl on
>> intel targets and it was
>> a fscking patch orgy. maybe that experience kept me from trying to build
>> it with uclibc yet.
> I've learned not to try to fix FSF code, I've learned to lie to it
> and/or bypass large chunks of it instead.
>
> You've learned never to install Libtool on Linux, right?

most stuff actually seems to come with its own libtool.
however i just found out that you have to delete *.la and magically 
stuff builds, that failed earlier.
>    http://landley.net/notes-2006.html#10-10-2006
>
> Libtool exists to make non-ELF systems work like ELF systems.  When used
> on a system that _is_ using ELF linking, it should be a NOP.
>
> Having nothing to do and still screwing it up is a registered trademark
> of the free software foundation.
>
>> the number of glibc-specific assumptions gdb makes is huge, not to talk
>> about *gnulib*,
>> which is broken by design, and of course is "built-in" into gdb.
>>>> musl had fcntl.h with values from i386, which slightly differ from
>>>> ARM, so
>>>> the O_LARGEFILE of musl's open() was interpreted as O_NOFOLLOW from the
>>>> kernel.
>>>> was fixed yesterday by
>>>> http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=e6d765a8b1278e9e5f507638ccdec9fe40e52364
>>>>
>>> Yeah that'll do it.
>>>
>>>> (that's also a nice thing about musl, i report the bug and 10 minutes
>>>> later it is fixed, usually)
>>> Cool.  I try to fix issues the same week they're reported (clearing my
>>> backlog on weekends), but don't always succeed.  (I have a day job and
>>> working on this isn't it.)
>> (heh, you still didn't fix the busybox "patch program" bug with path
>> levels i reported on the busybox mailling list.
>> thats why sabotage uses gnu patch now ;))
> Um, send me a link again?

http://lists.busybox.net/pipermail/busybox/2011-July/076293.html
> Rob
>
>


 1318051196.0