[Aboriginal] What's musl, anyway?

[Rob Landley]

On 10/05/2011 11:57 AM, James McMechan wrote:
>> > gcc 3.4.6 is a relatively nice compiler, builds with less than 128MB
>> > RAM, a statical linked crosscompiler fits into a 6MB .xz file,
>> > it's faster than gcc4, and has relatively good optimization, compared
>> > to pcc or tcc.
>>
>> Compared to tcc Turbo C for DOS had relatively good optimization.
> 
> tcc the turbo C from dos or tcc the tiny c compiler which you appear
> to be thinking of below (http://bellard.org/tcc).

I maintained a fork of that compiler for a couple years:

  http//landley.net/hg/tinycc

They still mention it from the tcc web page and tinycc wikipedia
articles, last I checked.

Fabrice more or less forked qemu off of the older tcc project, and then
that took over all his time the same way buildroot sucked away the
uClibc developers and buried discussion on its mailing list until the
project stagnated.

I put a lot of effort into getting uClibc unsquashed over the past few
years (creating the buildroot mailing list and kicking the discussion
over there, more or less appointing the current uClibc maintainer by
fiat, sometimes it's nice to have root access to a shared server...)

I was sort of trying to do the same thing to tcc but all I managed to do
was prompt Fabrice to hand maintainership over to a Windows developer,
who has no interest (or expertise) in Linux, and has no clue to manage
an open source project.  But it's "official" so I closed down my fork
and went off and did other things:

  http://landley.net/code/tinycc

I still need to chop out tcc's preprocessor and hook it up so distcc is
calling tcc -E instead of gcc -E, which might speed up distributed
compiles under qemu quite a bit.  (It's up there with making all the
targets use emulated gigabit ethernet instead of crazy old 10baseT cards
that are expensive to emulate, and letting the emulated boards allocate
more than 256 megs for their physical memory.)

I also still have vague plans to glue QEMU's Tiny Code Generator (tcg)
to the back of either tcc's front end or sparse, I made some notes about
this back in 2009:

  http://landley.net/code/tinycc/qcc/todo.txt
  http://landley.net/code/tinycc/qcc/commands.txt

(I actually had a mailing list for it at one point, but it stalled when
I got buried in other things and went away when I switched hosting
providers.)

>> > never heard about tcg. i'll read up on that one.
>>
>> There's a README in qemu's tcg subdirectory. See also
>> http://127.0.0.1/qemu/2008-01-29.html#Feb_1,_2008_-_TCG
> 
> Err, Rob this appears to be a archive on your machine...

Heh, sorry.  I have an apache instance running on loopback with a copy
of my website, and I look stuff up locally and then rename it when
posting the URL.  I sometimes miss them. :)

> http://landley.net/qemu/2008-01-29.html#Feb_1,_2008_-_TCG works
> is tcg better?

Very much so, yes.

> tcc also by Bellard has i386,x86_64,arm,c67? architectures

Fabrice Bellard is awesome:


http://www.softwarequalityconnection.com/2011/03/fabrice-bellard-portrait-of-a-superproductive-programmer/

But he essentially abandoned tinycc in 2005, to spend his time on qemu.
 (As far as I can tell, QEMU happened when he started doing so
multiple-backend cleanup on tcc and went "what if I had multiple
front-ends as well, that could handle input other than C code, such as
another target's assembly code", and he made a kickass dynamic
translator, and the rest is history.)

The arm and x86_64 targets happened after Fabrice left the project, and
are not well integrated (or debugged). x86-64 is eclipsing i386 but arm
didn't even have a libgcc.a replacement last I checked, they still told
you to copy gcc's.

Also, _which_ arm is that?  armv4l, armv4tl, armv5l, armv6l, armv7l?
The tcg code can do all that, plus both big and little endian, plus
several different floating point variants, plus mips (several different
processor variants, both endiannesses, both 32 and 64 bit), powerpc
(ditto), cris, microblaze, lm32, sparc, s390, sh4, m68k, alpha, and more
added all the time.

If you look at the tinycc release history, there was a multi-year gap
after 2005, and then a burst of effort in 2008 (when my fork spurred the
original project back to life), and then it's got 2 and 1/2 years
without a release since my fork stopped.  An active project does attract
additional developers, thus the arm code dated from the tail end of
Fabrice's tenure (but I'm the one who integrated it and made it work),
and the x86-64 code dated from the activity period my fork spurred
(although it was written against the "original" version, and was kind of
desperately overdue because the switchover to 64 bit PCs had happened 3
years earlier and I believe Intel had already stopped _making_ 32 bit
chips by that point, for desktops and laptops, anyway).

The thing to remember about tcc is that in 2004, Fabrice did tccboot:

  http://bellard.org/tcc/tccboot.html

That used a modified 2.4 kernel, only building a carefully selected
subset, and removing/rewriting features even from that which the
compiler couldn't deal with at the time, plus totally skippping dealing
with the makefiles at all.  But the point is, it WORKED.  (Still does,
it's fun to boot the iso under qemu.)

Keep in mind, tcc started as a JOKE in 2002 (http://bellard.org/otcc/)
and 2 years later it was building the Linux kernel.

Seven years have passed since that, and you can't even reproduce the old
tccboot with the current code, let alone build anything more recent, let
alone use tcc as an actually gcc replacement building a
linux+uClibc+busybox bootable system.

Any wonder why I want to leverage Fabrice's _newer_ design and the very
active community he built around it?

> I think, and so has code generators for the most common 3 cases.

The arm target is almost completely unused, and tcc didn't run natively
_on_arm_ due to alignment issues when I stopped my fork.  (Perhaps
they've fixed it since then, but I doubt they're even aware of the issue.)

> or it just the remaining arches like
> ppc32,ppc64,sparc32,sparc64?,mips32,mips64
> sh4 that is the concern. it also lacks gcc -Ml,d option for running the
> compiler
> to find out what the code being compiled depends on, that was one of the
> things
> linux kernel and busybox need gcc for :(

I hadn't noticed.  I got blocked by its lack of vararrays (c99 feature
the kernel depends on) and dead code elimination (kernel and busybox
both need it or you get link failures, again something turbo C for dos
did back in the 80's)

I was working to fix that, but compiling Linux packages is not on
grischka's radar, he's a windows developer.  And I got sick of competing
with a zombie following me around and having to port all external
contributions from their tree to mine, despite how far mind had
diverged, and despite the majority of the work in that tree being ports
of what I was doing...

*shrug*

>> Once upon a time I did create partitioned images:
>>
>> http://landley.net/code/mkhda.sh
>>
>> But it's extra work for no benefit, and it means you can't easily
>> loopback mount them from the host.
>>
>> > b) after unpacking and configuring gmp-5.0.2, i have a symlink
>> > "gmp-5.0.2/mpn/add_n.asm -> ../mpn/arm/add_n.asm"
>> > the symlink target is a regular file, but the readlink syscall returns
>> > ELOOP in errno.
> 
> This symlink is not present before configure, the style of ../mpn/arm
> is slightly odd. I would expect ./arm/add_n.asm to be simpler,
> even the x86_64 version mpn/add_n.asm -> ../mpn/x86_64/aors_n.asm
> is strange, why a symlink to a arch file of a different name?...
> The going up and then down the mpn directory may be hitting some
> oddness, though local ref symlinks usually work just fine.
> The ELOOP indicating too many symlinks in path usually only occurs
> when a symlink loop is found...
> Was it being built as part of the Aboriginal scripts or was it being
> built by hand?

Presumably it's being built natively under the resulting system
(probably running under QEMU).  That's one of the central design ideas
of Aboriginal Linux, "we cross compile so you don't have to":

http://speakerdeck.com/u/mirell/p/developing-for-non-x86-targets-using-qemu

Rob

 1317924295.0