[Aboriginal] What's musl, anyway?

[Rob Landley]

On 10/08/2011 12:19 AM, John Spencer wrote:

> here's a writeup from the amd64 port:
> http://brightrain.aerifal.cx/~niklata/PORTING
> 
> this commit here documents about everything needed for the arm port
> http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=d960d4f2cbf18ff3476c7ac03698ec253885dd8e
> 
> 
> and these two here add dynamic linking to musl arm
> http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=12ace5bf76040e928df1abd57e9b5134ca4647c6
> 
> http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=fcaf7065698bce12a444803587421ae88446f6c0
> 
> 
> see how little code was needed...

Cool.

I note that uClibc currently has no s390 support, but these days QEMU
_does_.  This means I can't support all the QEMU targets using uClibc
unless their development community gets fundamentally unstuck.

If musl supported that target and uClibc didn't, I'd logically want to
see about getting musl support into aboriginal itself.  (This would mean
at minimum building current busybox against it, plus an older GPLv2
release of a gnu toolchain (gcc 4.2.1, binutils-1.17, and make 3.81 were
the last GPLv2 releases, and bash 2.05b is "good enough" about about 1/3
the size of modern bash.)

The rest of the packages used during the aboriginal linux build are
distcc, e2fsprogs, genext2fs, and squashfs (which requires zlib).  (And
of course current linux but it doesn't depend on libc.)

>> (In aboriginal, all the information about a specific target is in the
>> sources/targets files, one per target.  At one point many moons ago i
>> had documentation about adding a new target.  um...
>> http://landley.net/aboriginal/documentation.html#new_target I think.
>> Wow that's out of date...)

I need at least a month to do nothing but documentation, but I know from
experience it wouldn't be enough.

>>>> This is different from the attempts to port bionic to the vanilla
>>>> kernel, has no relation to klibc, is not related to uClibc, dietlibc,
>>>> glibc, libc5, or any of the BSD libc projects?
>>>>
>>> yep, different attempt. and so far it's looking very sweet. couldn't do
>>> it better myself.
>>> completeness status is reflected in the version number which is 0.8.xx
>>> there's some C99 math stuff missing and no C++ support yet, but apart
>>> from that nearly everything compiles now
>> Why should libc care about c++ again?  C++ is not C, any more than
>> Objective C is.  Java borrowed a lot of C's syntax, and isn't C either.
>>
> 
> indeed, i actually would prefer a system without any C++ crap.

NO_CPLUSPLUS=1 does it for aboriginal, but people keep writing stuff in
it...

> however there are many useful apps written in it (like gold), and to get
> wide acceptance musl has to get some way to build C++ stuff. it's
> unlikely that the GCC guys will do it...

You're aware of uClibc++, right?  (My project builds it and all. :)
It's a bit stalled right now, but I have Garret's phone number around
here somewhere...

What's needed _in_ the libc?  Stack unwinding cruft?



>> In theory, the system images I built and posted have working c++
>> toolchains in them, based on uClibc+=.  Haven't stressed it much, but it
>> builds hello world with the funky "we redefined the bit shift operators
>> to perform I/O" stuff.
>>
>> (It died building cmake, I have to go back and work out why...)
>>
> 
> not really in the mood to fight with C++ code currently.

Neither am I.  That's why it's been on my todo list for over a year.

> btw, i could compile gcc 4.5.3 by passing -Wl,--no-keep-memory to the
> linker.
> i also added a patch which disables target-libiberty and target-libz.
> with disable-bootstrap the compiler built in around 15 minutes using
> distcc o.0
> so it actually looks as if --no-keep-memory actually is *faster* despite
> being described as "slow but less memory hungry"

The FSF remains crazy, although sometimes performance stuff is non-obvious:

http://www.funtoo.org/wiki/Funtoo_Filesystem_Guide,_Part_5#data.3Djournal_Mode

>>> i was told that it is faster and takes way less ram than gnu ld.
>>> (which is not very surprising, given the quality of average gnu code)
>> The tinycc linker is also buckets faster and less memory intensive than
>> gnu ld.
> 
> how about this one ? http://code.google.com/p/ucpp/

I haven't done a survey recently, but I do note the explosion of
projects since gcc went GPLv3 and stopped being "good enough" for Apple
or the BSDs or the Linux kernel developers or basically anybody except
the FSF.  I was working on tcc but there's clang and the new pcc and
open64 and some kernel guys are gluing sparse to llvm...

I expect gcc's days are numbered but displacing it is like displacing
windows: it's an entrenched standard zealously defended by egocentric
bastards out to impose Their Way upon the world, so there's work to do
getting rid of it:

http://flyingbynight.wordpress.com/2011/02/28/phnglui-mglw-nafh-cthuhu-riv-wgah-nagl-fhtag/

>> Too bad the maintainer of that project is a windows developer,
>> if it would just DIE I'd restart my fork but I refuse to fight zombies
>> without a cricket bat.
> 
> hmm iirc the mob branch is the only active. it's not even that bad, i
> checked it out lately and it built stuff which tcc 0.9.25 refused to build.

My tree also built a lot of stuff 0.9.25 refused to build.  That was
sort of the point of my tree.

>>>>> indeed, the more targets, the better. however i've seen in gcc's
>>>>> changelog that
>>>>> in the last releases some old architectures have been removed...
>>>> Did I mention I think the FSF is really really bad at software
>>>> engineering?
>>>>
>>> full ack. what sucks most tho are the build systems used by gnu.
>>> i guess to fully grok gcc's build system you have to become Phd at the
>>> gnu university.
>> http://landley.net/notes-2010.html#11-11-2010
>> http://landley.net/notes-2010.html#15-10-2010
>> http://landley.net/notes-2010.html#26-08-2010
>> http://landley.net/notes-2010.html#20-08-2010
>>
>> And so on, and so forth.  Trust me: they don't understand it either.
>> They just keep adding another special case for each new test case.
>>
>> Literally.  That is the design of autoconf.
>>
> 
> interesting lecture. i actually think it would make sense to writeup a
> "serious" blog post [web 2.0 like with comments and such] about
> autoconf, with some examples like these here and link it on reddit.

Eric Raymond and I had ideas for a paper on this, then we had a falling
out because his libertarian views got co-opted by the
very-merry-unbirthday crowd and now he's a climate change denialist, and
really I don't want to get any of that on me.  It's not the fun kind of
crazy anymore.

> compiling stuff could be so much fun if autoconf'd build systems
> wouldn't always fuck everything up.

Cross compiling would still suck.

> actually it is so broken that it defeats the purpose of the FSF, in that
> it makes it too hard to change and recompile your free software, making
> it unfree that way.

The FSF has been self defeating for many years.  If you want a RANT:

  http://landley.net/notes-2010.html#17-07-2010
  http://landley.net/notes-2010.html#19-07-2010

(First part is background necessary to understand the second part.  I
keep meaning to write a book on computer history, but I have too many
other things to do...)

>> You have to add said flag to the preprocessor as well, it sets a bunch
>> of symbols telling you about your current target.  Go:
>>
>>    gcc -E -dM -<  /dev/null | less
>>
>> Now do that for each of the cross compilers and diff the result. :)
> 
> ... thats quite a list.

Now ponder that list and go "hey, it tells me my current target and my
current endianness... what is half of ./configure for, anyway?"  Add in
"what is libtool for", or the fact that pkg-config is so obviously
stupid you can sum it up with stick figures http://xkcd.com/927/
(because rpm and dpkg and portage and pacman and bitbake and so on
weren't _enough_), and PKG_CONFIG=/bin/true is a beautiful thing...

>> I've learned not to try to fix FSF code, I've learned to lie to it
>> and/or bypass large chunks of it instead.
>>
>> You've learned never to install Libtool on Linux, right?
> 
> most stuff actually seems to come with its own libtool.
> however i just found out that you have to delete *.la and magically
> stuff builds, that failed earlier.

Once upon a time I meant to do a libtool-stub that actually WOULD be a
NOP successfully, like http://penma.de/code/gettext-stub/ and my little
trick here:

http://landley.net/hg/control-images/file/9/common/bootstrap/build-one-package.sh
(lines 31-41).

By the way, may I say that wordwrapping URLs is so far down on
thunderbird's list of "appalling incompetence" it barely registers?  I
REALLY need to get a new mail program.  (I just need a couple hours to
set up all my filters again...)

Anyway: I need to make a NOP libtool that I can symlink the others to,
which just converts the command line arguments to ld format and does
NOTHING ELSE.  A small shell script, probably.  Then ln -sf as needed.

>>>>> (that's also a nice thing about musl, i report the bug and 10 minutes
>>>>> later it is fixed, usually)
>>>> Cool.  I try to fix issues the same week they're reported (clearing my
>>>> backlog on weekends), but don't always succeed.  (I have a day job and
>>>> working on this isn't it.)
>>> (heh, you still didn't fix the busybox "patch program" bug with path
>>> levels i reported on the busybox mailling list.
>>> thats why sabotage uses gnu patch now ;))
>> Um, send me a link again?
> 
> http://lists.busybox.net/pipermail/busybox/2011-July/076293.html

Ah yes.  I'll try to get to it this weekend.

Rob


 1318094858.0