[Aboriginal] What's musl, anyway?

[John Spencer]

On 10/08/2011 07:27 PM, Rob Landley wrote:
> On 10/08/2011 12:19 AM, John Spencer wrote:
>
>> here's a writeup from the amd64 port:
>> http://brightrain.aerifal.cx/~niklata/PORTING
>>
>> this commit here documents about everything needed for the arm port
>> http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=d960d4f2cbf18ff3476c7ac03698ec253885dd8e
>>
>>
>> and these two here add dynamic linking to musl arm
>> http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=12ace5bf76040e928df1abd57e9b5134ca4647c6
>>
>> http://git.etalabs.net/cgi-bin/gitweb.cgi?p=musl;a=commitdiff;h=fcaf7065698bce12a444803587421ae88446f6c0
>>
>>
>> see how little code was needed...
> Cool.
>
> I note that uClibc currently has no s390 support, but these days QEMU
> _does_.  This means I can't support all the QEMU targets using uClibc
> unless their development community gets fundamentally unstuck.
>
> If musl supported that target and uClibc didn't, I'd logically want to
> see about getting musl support into aboriginal itself.
as it looks currently, you'd have to make that port yourself ;)
actually the arm port has just been finished and as it looks i'm 
currently the only using/testing it...
i'd expect mips/ppc support coming next, *if* some other ports will 
happen in the near future.
> (This would mean
> at minimum building current busybox against it, plus an older GPLv2
> release of a gnu toolchain (gcc 4.2.1, binutils-1.17, and make 3.81 were
> the last GPLv2 releases, and bash 2.05b is "good enough" about about 1/3
> the size of modern bash.)
make 3.81 has the additional advantage that it builds older kernels as 
well, since some breaking change has been introduced with make 3.82 (for 
example linux 2.6.25 won't build on 3.82)
the binutils however are too old to play nice with musl.

> The rest of the packages used during the aboriginal linux build are
> distcc, e2fsprogs, genext2fs, and squashfs (which requires zlib).  (And
> of course current linux but it doesn't depend on libc.)
is aufs enabled as well ? i thought about adding support for that to my 
build manager, as it turned out that decompressing the kernel each time 
i build it eats a lot of time (about 10 minutes in qemu). doing the 
untaring in a separate step and overlay it with a build-only directory 
would allow simply trashing the built stuff and restart clean without 
having to untar again (since "make clean" won't remove patches, and so on).
>>> (In aboriginal, all the information about a specific target is in the
>>> sources/targets files, one per target.  At one point many moons ago i
>>> had documentation about adding a new target.  um...
>>> http://landley.net/aboriginal/documentation.html#new_target I think.
>>> Wow that's out of date...)
> I need at least a month to do nothing but documentation, but I know from
> experience it wouldn't be enough.
>

the aboriginal infrastructure is indeed pretty sweet and i was thinking 
about wether i should adopt parts of it.
especially the automatic distcc configuration, although the 
preprocessing power needed for some stuff makes it nearly useless in 
those cases. if that bottleneck could be removed, it would even make 
sense to allow to add more distcc-servers.
well, maybe some day a sane person (which happens to know python) will 
come and port the distcc pump stuff to C, or just rewrite it from scratch.
>>>>> This is different from the attempts to port bionic to the vanilla
>>>>> kernel, has no relation to klibc, is not related to uClibc, dietlibc,
>>>>> glibc, libc5, or any of the BSD libc projects?
>>>>>
>>>> yep, different attempt. and so far it's looking very sweet. couldn't do
>>>> it better myself.
>>>> completeness status is reflected in the version number which is 0.8.xx
>>>> there's some C99 math stuff missing and no C++ support yet, but apart
>>>> from that nearly everything compiles now
>>> Why should libc care about c++ again?  C++ is not C, any more than
>>> Objective C is.  Java borrowed a lot of C's syntax, and isn't C either.
>>>
>> indeed, i actually would prefer a system without any C++ crap.
> NO_CPLUSPLUS=1 does it for aboriginal, but people keep writing stuff in
> it...
>
>> however there are many useful apps written in it (like gold), and to get
>> wide acceptance musl has to get some way to build C++ stuff. it's
>> unlikely that the GCC guys will do it...
> You're aware of uClibc++, right?  (My project builds it and all. :)
> It's a bit stalled right now, but I have Garret's phone number around
> here somewhere...
>
> What's needed _in_ the libc?  Stack unwinding cruft?
>
>

from what i've heard there's special atexit/dynlink stuff needed and 
partially some symbol names clash.
currently there's some discussion about that on the musl mailinglist.

>>> In theory, the system images I built and posted have working c++
>>> toolchains in them, based on uClibc+=.  Haven't stressed it much, but it
>>> builds hello world with the funky "we redefined the bit shift operators
>>> to perform I/O" stuff.
>>>
>>> (It died building cmake, I have to go back and work out why...)
>>>
>> not really in the mood to fight with C++ code currently.
> Neither am I.  That's why it's been on my todo list for over a year.
>
>> btw, i could compile gcc 4.5.3 by passing -Wl,--no-keep-memory to the
>> linker.
>> i also added a patch which disables target-libiberty and target-libz.
>> with disable-bootstrap the compiler built in around 15 minutes using
>> distcc o.0
>> so it actually looks as if --no-keep-memory actually is *faster* despite
>> being described as "slow but less memory hungry"
> The FSF remains crazy, although sometimes performance stuff is non-obvious:
>
> http://www.funtoo.org/wiki/Funtoo_Filesystem_Guide,_Part_5#data.3Djournal_Mode
>
indeed.

>>>> i was told that it is faster and takes way less ram than gnu ld.
>>>> (which is not very surprising, given the quality of average gnu code)
>>> The tinycc linker is also buckets faster and less memory intensive than
>>> gnu ld.
>> how about this one ? http://code.google.com/p/ucpp/
actually that is a *preprocessor*... we were talking about ripping out 
the tcc preprocessor somewhere else...
> I haven't done a survey recently, but I do note the explosion of
> projects since gcc went GPLv3 and stopped being "good enough" for Apple
> or the BSDs or the Linux kernel developers or basically anybody except
> the FSF.  I was working on tcc but there's clang and the new pcc and
> open64 and some kernel guys are gluing sparse to llvm...
>
> I expect gcc's days are numbered but displacing it is like displacing
> windows: it's an entrenched standard zealously defended by egocentric
> bastards out to impose Their Way upon the world, so there's work to do
> getting rid of it:
>
> http://flyingbynight.wordpress.com/2011/02/28/phnglui-mglw-nafh-cthuhu-riv-wgah-nagl-fhtag/
>
i was looking lately at the tendra compiler, which has just been 
uploaded to https://bitbucket.org/asmodai/tendra/
that seems like a well designed compiler with good optimization potential.
it uses some internal intermediate format (which is even standardized), 
which then gets optimized according to the target.
however the build system is pretty outdated and i was unable to 
bootstrap it in its current state.
i hope some activity on that project will start soon...
>>> Too bad the maintainer of that project is a windows developer,
>>> if it would just DIE I'd restart my fork but I refuse to fight zombies
>>> without a cricket bat.
>> hmm iirc the mob branch is the only active. it's not even that bad, i
>> checked it out lately and it built stuff which tcc 0.9.25 refused to build.
> My tree also built a lot of stuff 0.9.25 refused to build.  That was
> sort of the point of my tree.
really unfortunate that you stopped it. but i have read your statement 
and respect your reasons.
in the meantime, an uncoordinated "mob" drives the project on repo.or.cz ;)

>>>>>> indeed, the more targets, the better. however i've seen in gcc's
>>>>>> changelog that
>>>>>> in the last releases some old architectures have been removed...
>>>>> Did I mention I think the FSF is really really bad at software
>>>>> engineering?
>>>>>
>>>> full ack. what sucks most tho are the build systems used by gnu.
>>>> i guess to fully grok gcc's build system you have to become Phd at the
>>>> gnu university.
>>> http://landley.net/notes-2010.html#11-11-2010
>>> http://landley.net/notes-2010.html#15-10-2010
>>> http://landley.net/notes-2010.html#26-08-2010
>>> http://landley.net/notes-2010.html#20-08-2010
>>>
>>> And so on, and so forth.  Trust me: they don't understand it either.
>>> They just keep adding another special case for each new test case.
>>>
>>> Literally.  That is the design of autoconf.
>>>
>> interesting lecture. i actually think it would make sense to writeup a
>> "serious" blog post [web 2.0 like with comments and such] about
>> autoconf, with some examples like these here and link it on reddit.
> Eric Raymond and I had ideas for a paper on this, then we had a falling
> out because his libertarian views got co-opted by the
> very-merry-unbirthday crowd and now he's a climate change denialist, and
> really I don't want to get any of that on me.  It's not the fun kind of
> crazy anymore.
>
>> compiling stuff could be so much fun if autoconf'd build systems
>> wouldn't always fuck everything up.
> Cross compiling would still suck.
>
>> actually it is so broken that it defeats the purpose of the FSF, in that
>> it makes it too hard to change and recompile your free software, making
>> it unfree that way.
> The FSF has been self defeating for many years.  If you want a RANT:
>
>    http://landley.net/notes-2010.html#17-07-2010
>    http://landley.net/notes-2010.html#19-07-2010
>
> (First part is background necessary to understand the second part.  I
> keep meaning to write a book on computer history, but I have too many
> other things to do...)
>

the mail from rms about "making the code hard to use" to get evil 
companies out is indeed shocking.
it seems GNU software sucks by design.

>>> You have to add said flag to the preprocessor as well, it sets a bunch
>>> of symbols telling you about your current target.  Go:
>>>
>>>     gcc -E -dM -<   /dev/null | less
>>>
>>> Now do that for each of the cross compilers and diff the result. :)
>> ... thats quite a list.
> Now ponder that list and go "hey, it tells me my current target and my
> current endianness... what is half of ./configure for, anyway?"

indeed.

> Add in
> "what is libtool for", or the fact that pkg-config is so obviously
> stupid you can sum it up with stick figures http://xkcd.com/927/
> (because rpm and dpkg and portage and pacman and bitbake and so on
> weren't _enough_), and PKG_CONFIG=/bin/true is a beautiful thing...
>

PKG_CONFIG=/bin/true<- that actually works ?


>>> I've learned not to try to fix FSF code, I've learned to lie to it
>>> and/or bypass large chunks of it instead.
>>>
>>> You've learned never to install Libtool on Linux, right?
>> most stuff actually seems to come with its own libtool.
>> however i just found out that you have to delete *.la and magically
>> stuff builds, that failed earlier.
> Once upon a time I meant to do a libtool-stub that actually WOULD be a
> NOP successfully, like http://penma.de/code/gettext-stub/ and my little
> trick here:
>
> http://landley.net/hg/control-images/file/9/common/bootstrap/build-one-package.sh
> (lines 31-41).

"Autoconf is useless" ;)

> By the way, may I say that wordwrapping URLs is so far down on
> thunderbird's list of "appalling incompetence" it barely registers?  I
> REALLY need to get a new mail program.  (I just need a couple hours to
> set up all my filters again...)
>
> Anyway: I need to make a NOP libtool that I can symlink the others to,
> which just converts the command line arguments to ld format and does
> NOTHING ELSE.  A small shell script, probably.  Then ln -sf as needed.

i shall appreciate it as well.

>>>>>> (that's also a nice thing about musl, i report the bug and 10 minutes
>>>>>> later it is fixed, usually)
>>>>> Cool.  I try to fix issues the same week they're reported (clearing my
>>>>> backlog on weekends), but don't always succeed.  (I have a day job and
>>>>> working on this isn't it.)
>>>> (heh, you still didn't fix the busybox "patch program" bug with path
>>>> levels i reported on the busybox mailling list.
>>>> thats why sabotage uses gnu patch now ;))
>>> Um, send me a link again?
>> http://lists.busybox.net/pipermail/busybox/2011-July/076293.html
> Ah yes.  I'll try to get to it this weekend.

thanks for the patch. will test it as soon as the kernel compile is 
finished...
> Rob
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal-landley.net/attachments/20111012/b7189fbf/attachment-0003.htm>