[Aboriginal] Is moving control-images to their own repository a good idea?

[Rob Landley]

On 07/03/2011 06:52 AM, Bjørn Forsman wrote:
> 2011/7/3 Rob Landley <rob at landley.net>:
>> On 07/01/2011 10:26 AM, Bjørn Forsman wrote:
>>> On 1 July 2011 14:50, Rob Landley <rob at landley.net> wrote:
>>>> I've been thinking of moving sources/control-images to its own project
>>>> for a while now, and am poking at it this morning.
>>>
>>> Although I see the point in separating the two, because it is "clean",
>>> how about keeping them with aboriginal a little longer, for the sake
>>> of user + developer(?) convenience?
>>
>> What use case did you have in mind?
> 
> [Disclaimer: I don't regularly use aboriginal, but I find it very
> interesting and would like to start using it to bootstrap stuff.]
> 
> For users: only one download, and having control-images in the same
> repo makes them "official" and can make it easier to regression test
> aboriginal.

I'm all for having an official collection, but needing to get them
upstream into the aboriginal repo isn't necessarrily the best way to do
that.

Aboriginal Linux and the control images do different things.  The first
provides a generic build environment, the second builds things in that
environment.  The first project is essentially feature complete, and at
this point the changes to it are adding support for more targets and
upgrading the package versions, but not really adding a whole lot of new
_features_.  (Maybe v9fs/virtio server support to talk to the host
without an ftp server, or rewriting distcc with something simplaer and
more reliable and controllable.  But that's "doing what we already do a
better way" rather than really doing something new.)

The control images are just starting.  I have no idea what people will
make control images to _do_, and what I want to get them to do is a very
long unfinished list: gentoo support's maybe 1/3 done, LFS needs and
upgrade and to work on all targets, I need to add bootstraps for fedora
and debian and slackware and arch and who knows what else, I need to
simplify and cleanup the bootstrap-skeleton stuff...

One of the most important things about project design is knowing what
your boundaries are.  You can't answer "what does this project do"
without being able to say what it does NOT do.  Where does this project
STOP?

Aboriginal Linux's major design goal is in our motto: "we cross compile
so you don't have to".  It does all the nasty bootstrapping bits to get
you a native build environment, so you can transition to native
development.  (On target hardware, under an emulator, up to you.)
That's what we do.

What we DON'T do is be a distro.  That's the trap buildroot fell into,
and when the openembedded guys got sick of buildroot and started over
from scratch they dove right back into tha trap.  We try to keep "what
you build" and "how you build" separate.  What targets you want to
support and what packages you want to build should be separate decisions.

Because of this, we don't cross-compile more packages than absolutely
necessary.  (Even distcc is pushing it, since it _could_ be build
natively like dropbear and strace are.  But it requires host setup, so
it crosses the host/target barrier, so it can't be cleanly encapsulated
within the target in a host-agnostic way, so...)

This means the development environemnt aboriginal linux produces is as
sparse as we can make it.  You can then natively work your way up to a
full RPM or .deb or portage environment and use an existing repository
which has a full-time team maintaining it.  Or take the LFS approach of
building and installing prerequisite packages by hand.  But both of
these are nontrivial, and it's nice to provide examples of how to do it.

The problem to be solved there (by the bootstrap-skeleton stuff) is
"Fedora wants to bulid under Fedora, Debian wants to build under Debian,
Gentoo wants to build under Gentoo".  Building a Fedora root filesystem
under Gentoo is an unsolved problem which is not properly documented
anywhere.  It should be.  But that's not Aboriginal Linux's problem,
it's a separate project.

But bootstrapping distros isn't the only use of control images: you can
also regression test package du jour, with automated builds from a cron
job or similar, or running the busybox test suite (or LTP) under the
target.  Examples for how to do this are useful too, showing how to
Aboriginal Linux's "automated, under a cron job" infrastructure.

This collection of control images is unbounded.  There's no design limit
on what you put in there (not yet, anyway).  If they're part of the
Aboriginal Linux repository, then there's no limit on what we can check
into that repository...

I'm not falling into the distro trap through the backdoor either. :)

> For developers I was going to say that it makes it easy to do changes
> that affect the interface between the control-images and the system
> image, but as you point out below, this is not an issue:

It is an issue, but I explicitly _want_ to keep them separated.  I want
this interface to be stable and general purpose.  To do that, it must
remain agnostic about what's going to be using it.

You can do anything in a control image.  I'm not going to pretend I can
even imagine the kind of things you can do, it's a general purpose Linux
root filesystem you boot and run stuff in.

>> The native build infrastructure doesn't care where the control image
>> came from, you just point it and go.  (You can download a system image
>> and a control image and set them on each other without compiling
>> anything on your host.)
> 
> IMHO, until the control-images in aboriginal starts to cause a lot of
> churn, it is probably easier for users and developers to keep them in
> the same repo. Although aboriginal is supposed to be the *minimal*
> build environment that can bootstrap itself, having one extra stage
> that builds a minimal gentoo/lfs/debian/... is a major boon, IMHO.

I think it's very worth doing, and it's pretty much where I'm turning my
attention.  Aboriginal Linux itself is feature complete as of 1.0.  It
does what it set out to do, it just needs package upgrades and more
targets.  Now the action moves to the control images.

But that pretty much means finishing one project and starting another.
Providing a build environment and using that build environment are
different projects.  Aboriginal Linux is _simple_ (or at least as simple
as I know how to make it given the job it's doing), and I don't want it
to stop being simple.

All I'm proposing doing is adding another entry under
http;//landley.net/hg and maybe a new web page.  I don't expect enough
traffic to justify a separate mailing list any time soon, though, that
can stay here. :)

Rob