[Aboriginal] new user on the list (with questions)

[Rob Landley]

On 04/07/2011 01:33 AM, Bjørn Forsman wrote:
> Hi Rob,
> 
> My name is Bjørn and I'm new on this list (have been lurking here for
> a few weeks though). I have been doing embedded linux systems for
> about three years. I'm a linux addict :-)

Hi Bjorn!  (Dunno how to do the 0 yet.)

> Most of the time I use Buildroot. But lately I've been wanting to use
> a "real distro" instead of an "accidental distro" (I've been reading
> your presentation.pdf and website :-) ). Unfortunately, it seems most
> "real" distros don't really care about being able to easily bootstrap
> themselves from source (as far as I can see). This is bad news for me,
> because I think being able to build from source (with a specific
> toolchain) is very important. I don't want to rely on binary stuff I
> cannot rebuild.

Linux From Scratch is the only "bootstrap from source" demo, and that
was easy-ish to automate (but still needs more testing, and more bugs
hammered out of the C library and configuration on non-x86 targets).

Gentoo  has an emphasis on building from source, but it expects you to
build Gentoo under Gentoo, not under another distro.  (So "build" not
quite equal "bootstrap".)  I have about half of a gentoo-bootstrap done,
which has been starved for attention for a while now.

Lots of distros have a root filesystem creator, something that populates
a chroot with a new root filesystem.  In debian the package is
"debootstrap", in gentoo it's "catalyst", in Red hat there's "Koji", and
so on.  (I have the SuSE and ubuntu ones written down around here
somewhere.  Oh, and Red Hat has a wrapper around Koji, but I have yet to
quite work out what it does.)

Unfortunately, these root filesystem creators generally work from their
binary repositories, not their source repositories.  They install a set
of prebuilt binary packages into the new chroot, until they have
something you can chroot into and run (plus dev tools until it's capable
of rebuilding itself, ad nauseum).

So what I've been doing is:

A) Find their package management system.  portage, rpm, dpkg, etc.

B) Figure out what's involved in building packages through said package
management system.  Red hat has "source rpms", gentoo has emerge, etc.
Sometime's there's reasonable documentation (such as
http://www.debian.org/doc/maint-guide/start.en.html#needprogs ) and
sometimes you just have to try it and see why it fails.

C) Try to cobble together a root filesystem in which their build runs,
and document the steps in a reproducible build script.

D) Build and test lots of packages to shake the bugs out.  (Having a
known working Linux From Scratch environment helps here, once you fix a
bug in uClibc and such it should be fixed for all distros...)

I've actually been meaning to take openembedded and buildroot and such
and treat them as small distros: try to chop off their "what you build"
and "how you build" bits to convince them to natively build their
packages on an unknown target.  I.E. treat buildroot like koji or
catalyst: something that populates a chroot using the host's native
toolchain.

Although technically those all want to install a new toolchain into the
chroot.  Which is sort of ok, except:

A) providing a working toolchain and libc on a given host isn't trivial.
 (The point of native compiling is you've already _got_ that, not that
it was easy to achieve, or to reproduce using different package versions
in who knows what configuration).

B) It breaks the distcc trick.  Not fatal, but very convenient if you
can keep that going.

> So, I'm really glad to find the Aboriginal Linux project!
> 
> I have some questions:
> 
> * Where is the hg clone URL for Aboriginal Linux?

hg clone http://landley.net/hg/aboriginal

> BTW, If you're considering setting up a git repo (I saw an email the
> other day), maybe this helps: http://hg-git.github.com/

A mirror.  I'd still apply new changes to the hg repo, but I could
cherry pick them out of a git repo if somebody else had one.  (Not sure
how to convince it the changes were the same for merge purposes, depends
on the clone tool.)

Looks like a nice tool.  I've bookmarked it, and will try to make time
to look at it properly this weekend.  Thanks.

> * Any chance Aboriginal Linux will support eglibc toolchains in the
> future? Or any other libc than uClibc? Or is this a non-issue after
> the next uClibc release as it will be possible to build other libc's
> with the uClibc toolchain inside the emulator?

That last one is the design goal.  I don't want to cross compile more
than one setup (eglibc is basically a reskinned glibc rather than a
separate project, and building glibc requires perl), but once uClibc has
TLS then glibc should build under it.  And THAT I want to have a
control-image to build.  (I'll might update the Linux From Scratch thing
to actually build the full toolchain, especially if binutils and gcc
will autodetect the host properly.  Yeah, I know I said that's a pain
with catalyst and such, but _those_ things are full of baked-in
assumptions that the distro they're bootstrapping is only ever built
under itself.  LFS has the whole chapter 5 airlock step.)

> * What do you think about crosstool-NG? (It already has
> uClibc/eglibc/glibc/newlib toolchains.) Could Aboriginal leverage the
> work of crosstool-NG?

Yup.

My build is a series of orthogonal stages, each designed to be replaced.
 The simple-cross-compiler, cross-compiler, and native-compiler stages
should all be disposable if you've got something else.  (You can just
untar it into the build directory, fix up the names, and run the next
stage.  You may have to touch the tarball name for build.sh to accept
that it's already there, that thing does dependency checking by seeing
if the previous stage's tarball is there or not.  Or just skip build.sh
and run the stages yourself, it's intentionally a trivial wrapper.)

> * Would you accept patches for an archlinux-bootstrap?

Yes please. :)

I spoke with the guy running the arch linux booth at scale, but
unfortunately he was not (remotely) an expert.  Couldn't really answer
any of my questions...

> It is currently
> on my todo because Archlinux seems like a good "embedded" distro: it
> has a small and efficient package manager written in C, works with
> binary package repositories, the build scripts / package metadata are
> simple shell scripts (easy to bootstrap) and the distro adheres to the
> KISS principle. Any thoughts on that?

It's already on my todo list, but several places down.

At the top of my todo list is finishing gentoo-boostrap because A) I've
already done half the work for that (and 2/3 of the research), B) it
builds from source already.

After that probably Debian/Ubuntu, becuase thats what I use on my laptop
and Ubuntu is the most popular Linux distro with somewhere around 40%
market share.  (And considerably higher among developer workstations.)

Red Hat had 50% market share among all Linux installs back when Red Hat
9 shipped, but then they figured out how to steal Sun's air supply.
Solaris' core market was large corporate and government procurement
contracts, which "protected" themselves by capping a contractor's
allowable profit margin at a percentage of the cost of materials.  So
the bidders specced the most expensive materials they could: instead of
a $30 Red Hat seat they specced a $5000 Solaris seat, because they'd
make $500 instead of $3.  When Red Hat figured this out and came out
with "Red Hat Enterprise" (I.E. "Pointy Hair Linux") they suddenly had
$100 million in annual orders from the fortune 500 (which technically
preferred Linux but wouldn't leave money on the table).  This WRENCHED
Red Hat's attention away from the desktop market (where they were making
maybe $10-$20 million annually), they discontinued it, renamed Rawhide
to "Fedora" (because their enterprise versions still needed R&D)...

Ahem.  Digression. :)

> * It may seem that Aboriginal Linux has a very small community (little
> ML traffic, all changes are done by you(?)), why do you think that is?

Really bad habits on my part?

  http://landley.net/aboriginal/history.html

This project has been one of my main hobbies for most of a decade now,
but until recently it was more exporatory/experimental than practical.
It's been relaunched from scratch three times (as I figured out "that's
not how I should be doing this"), and spun off a lot of tangents that
distracted me (yes, becoming busybox maintainer was a _tangent_ from
this project).

The problem for many years was I didn't have a clear idea what I was
trying to do, and didn't know how to do it anyway.  I was reacting
AGAINST the horrible bloated crappy GNU software, and the fact that my
first Linux machine ran great in 8 megs of ram but five years later you
couldn't even BOOT in 16 megs.  That was wrong and I wanted to stop it,
I just wasn't sure _how_.

As an example of one such shifting goal: after Red Hat got out of the
distro business (and Fedora Core 2 was such a disaster I declared it
unusable and started looking for other distros to use), I ran knoppix on
my laptop for a while.  Circa 2003 it was the ONLY distro even _trying_
for Linux on the desktop.  (I'm serious: I couldn't find anything else.
 I looked.  SuSE was going through bankruptcy, the Slackware guy had
some sort of terminal disease, Debian's infighting had completely
paralyzed the project, the gentoo people REFUSED to have anything to do
with installers because installing via extensive howto was considered a
rite of passage...  Lots of once-big distros like TurboLinux, Caldera,
Mandrake, and Xandros completely collapsed...  Ubuntu didn't ship its
first release until October 2004, and that first release was crap.)

I thought knoppix was great, but it used glibc and the gnu tools, which
right there took up maybe 10% of the whole CD.  Wouldn't a bootable CD
based on uClibc and busybox make more sense, and save more space for
other stuff?  (That was my vague end-goal in the 2003-2005 range, before
QEMU happened and I discovered cross compiling.)

It's hard to recruit other people to help you out with _design_ issues,
and with project goals.  I knew I wanted to make busybox into a
replacement for the gnu tools, and uClibc a replacement for gcc, and get
the FSF's "gnu/gnu/gnu/dammit we take credit for everything" hands off
of Linux.  But although that was an invariant all along, it was also
incidental.  I was always reaching for some goal beyond that.  (Such as
the current "make cross compiling go away", "make diverse Linux
platforms work the same", "perform automated cross-platform regression
testing", and so on.  The presentation listed several of them.  You'll
notice a theme here: what should we NOT be doing.  How can I demonstrate
that we don't NEED to do this?  That embedded development doesn't need
to mix cross compiling with package selection, for example...)

The project's current set of design goals tightened up around 2008 or so
(writing up everything I could think of to make the presentation.pdf
helped a lot, documenting stuff really does clarify your thoughts), and
september 2010 I got close enough to them to cut a 1.0 release that more
or less does that, from the right angle, with a tailwind.

The project's first short-lived mailing list wasn't until 2007 or so,
and this one that just restarted is its... third?  Fourth?  I never
really encouraged people to use the list instead of emailing me directly
so the majority of the traffic was always people emailing me instead of
people emailing the list.  (Horribly bad habit, I know.)

My blog (http://landley.net and http://landley.livejournal.com before
that) were the big "what have I been doing on this project" things
(updated daily for several years, http://landley.net/notes-2008.html has
347 daily entries out of 366 days in that year), but the blogs on
landley.net have no way to comment on them.  (More convenient for me,
less convenient for others.)

I became BusyBox maintainer for a year and change because of this
project (which essentially shut down work on FWL/Aboriginal because I
didn't have time).  When I started trying to substitute busybox for the
gnu tools it was a crazy thing to do that didn't remotely work: I spent
years _making_ it work, starting by rewriting "sed" until I could
substitute the busybox sed for gnu sed in the Linux From Scratch build
and the result actually worked right for all packages.  Then it was
replacing one command at a time and fixing each one.  I left busybox
because of bad blood between me and Bruce Fscking Perens, which
prevented me from upgrading busybox for a couple years and instead
working to reimplement it ala http://landley.net/code/toybox (I got over
it in 2008, commit 358, but that was another year or so of effort
diverted from working on this project).

There were other distractions.  I spent a year or so forking and
developing tinycc instead of focusing on this.  (I stopped when I
stimulated the original tinycc project into coming out of retirement,
Fabrice handed off maintainership to a windows developer and it became a
primarily windows project, not very interesting but still not something
I want to compete with.)  The uClibc project stalled horribly for years
and I had to do some political maneuvering to get that unstuck last year...

Anyway, I decided to get serious about it in 2008, and even start an
embedded consulting company so I could spend more time working on it and
less on other things.  That was a bad idea: between the recession and
running a company being a huge time sink, Mark and I folded that the
following year (and several months later when Mark wiped the server
running the domain I still hadn't migrated the project off yet; oops.
Lost the old mailing list.)

These days my day job is doing Linux kernel work for parallels, for the
past few months fighting with the NFS code to make it work in
containers.  Which is interting but again takes time away from working
on this.

Now that I have the 1.0 release out, the work is:

A) Regression testing against each new package release.  (Every time I
upgrade the kernel, or busybox, or uClibc, or qemu, or anything
really... it breaks some target somewhere.)

B) Making more targets work.  (Including fixing things like sh4, mips64,
and sparc that sort of work, or used to work, but can't build Linux From
Scratch.)

C) Cleanly separating the orthogonal layers so you _can_ swap in other
toolchains and such, or use this system-image script to package up
somebody else's root filesystem, or...

D) Documenting everything properly.  (I keep meaning to do a series of
podcasts explaining stuff, I haven't made the time and haven't got
decent recording equipment anyway.)

E) Then next big project on top of this: system images and bootstraping
distros.  (Which probably _is_ a separate project, and may need its own
repository.)

By the way, I'm currently stuck on the last gplv2 release of binutils
and gcc the same way I was stuck on busybox 1.2.2.  (I'm never going to
contribute to any GPLv3 licensed project unless paid, and I'm not
comfortable shipping binaries of that stuff.)  Unfortunately that means
I can't do armv7 and the microblaze targets.  I'm subscribed to the pcc
mailing list and I check in on llvm/clang from time to time, and I
really hope one of those unseats gcc.  (That's why I was working on
tinycc, but Grischka's permanently rendered that project irrelevant.)

Using crosstool-ng would be a great workaround for that.  They do
toolchains so I don't have to.  And if ALL they do is toolchains: great,
leverage it.  (But I need cross and native, and I'm picky about 'em, and
I still need to build my own C library because uClibc ain't fully cooked
yet, and ccwrap.c is a better approach than trying to get gcc's path
logic to _EVER_ work...)

> Thanks for writing Aboriginal Linux, I'm very excited about it :-)
> 
> Best regards,
> Bjørn Forsman

Woot.  Thanks.  Let me know what I can do to help. :)

Rob

 1302167636.0