[Aboriginal] greetings!

[Rob Landley]

On 07/12/2012 09:24 AM, David Henderson wrote:
> Wow and I often complain about my schedule! :)  Seems like you're
> working on 28 hour work days! lol  This might be a good opportunity for
> you to possibly offload some work then - at least on the distro
> creation/maintenance side.

Apparently, a chunk of messages from a month ago got missed. Catching up...

(It's possible I already answered this one, but if so I don't remember it.)

> My replies are in-line below to prevent confusion...
...
>     Define "their work"?
> 
>     I did "boot to a shell prompt on serial console, with native development
>     tools", targeting various qemu platforms. This let people build code
>     natively.
> 
>     I didn't really tackle "what tools do you need on target", because
>     everybody's needs are different and if you can build it natively, you've
>     got wget and a toolchain.
> 
>     The proliferation of boards is kind of amazing. A half-dozen projects
>     from buildroot to OpenEmbedded accumulated buckets of board
>     configurations, but never cleanly separated them out from toolchain and
>     root filesystem config, and I decided the correct thing to do was get
>     defconfigs upstream into linus's git.
> 
> Their "work" would be anything the user normally does during the day:
> email, word processing, listen to music, watch videos, program, etc. 

This is not a bounded set. Different people do different things.

> While the distro won't contain all of the software for everyones "work"
> (e.g. CAD users), it'll start with a set of software to perform the most
> common tasks.

Unfortunately the "most common tasks" really isn't a bounded set either.
People are strange.

> As time elapses, more and more can be added to include
> users with more rare "work".  This software, however, will be installed
> with their "personal data" as mentioned below.

Good luck with it. I think that's a can of worms.

>     > 2. Although the OS can be installed and run from a storage device,
>     it's
>     > intended to take a minimal amount of memory as it's home - RAM based.
>     > This should make upgrading as simple as replacing a single file!
> 
>     I remember ten years ago working out how to combine a kernel and root
>     filesystem into a single file, and patching lilo to have a "length"
>     parameter so it would only load the kernel and not the whole root
>     filesystem into memory.
> 
>       http://landley.net/aboriginal/old
> 
>     I even worked out some unused space in the ELF header (for user mode
>     linux, the emulator I was using at the time) which was the old floppy
>     boot sector on real kernels so either way "unused space" that I could
>     stamp a 4 byte length to say what offset losetup should use creating the
>     loopback file for the root filesystem. (I used zisofs because squashfs
>     wasn't merged yet.) Getting an initrd built into the kernel so it could
>     do the loopback setup and hand off is what got me poking at initramfs in
>     the first place...
> 
>     Good times, good times...
> 
> 
> Wow thats pretty interesting!  Currently XiniX uses grub, which I don't
> think has an option like that.

Neither did lilo, hence
http://landley.net/aboriginal/old/lilo-length.patch

>  Also, XiniX has the "firmware" and
> Kernel separated (I didn't even know those two could be combined).

Firmware turns out not to be a well-defined term, but between initramfs
and Greg Ungerer's "Linux without a bootloader" (slides
http://elinux.org/ELC_2010_Presentations video at
http://free-electrons.com/blog/elce-2010-videos/ ) there's no reason you
_can't_ do such a thing today. But it would be fairly hardware-specific.

> I suppose the integration of the two can help maintain system stability
> since everything is tested and released together...

Not necessarily. Having orthogonal modules you can test independently
and swap out different implementations of can help make the system work
better too. Otherwise you get horrible version skew instability like the
way udev only ever works with specific kernel versions.

> Did you ever run
> into any negative aspects when working like this?  The only one I can
> see off the top of my head is that if the distro isn't updated
> routinely, the kernel will get dated and present exploitable bugs to
> crackers.

If you're allowing arbitrary third parties to run random userspace code
on your system, it needs to be administered. But if all it's got is a
simple web server serving static pages out of a chroot there may not
actually _be_ any way to exploit it.

But the packaging mechanism was only mildly interesting, and I stopped
doing it years ago because the bootloader built into qemu was a better
solution for what I was trying to do.

>     > 3. Having an easily usable interface that's common among the various
>     > devices while being as simplistic and efficient as possible.
> 
>     Shell prompt?  xfce?  Web gui for headless boxes?
> 
>     Just getting the _drivers_ right for a wide variety of boards was a huge
>     pain for me, and I was focusing on qemu. And I have to regression test
>     them every kernel release and bisect some really nasty stuff sometimes.
>     (I'm doing a writeup the most recent round of git bisect to teach people
>     how to bisect bugs under adverse conditions. The one that broke armv5l's
>     scsi driver was buried several bugs deep behind "the serial console
>     broke", "build break", and so on...)
> 
> Currently XiniX is CLI only, but hopefully by the end of next month the
> graphical side will be up and running.

So it's been a month.  How did that go?

> In order to keep the size down
> while maintaining flexibility, the GUI is entirely web-based so it works
> with head and headless devices in a uniform, consistent way.  Also it
> does have software such as QT and/or GTK installed for software that
> currently doesn't have a web-interface-only alternative (and is also
> required for the webkit backend).

Back up: what problem are you trying to solve?

With Aboriginal Linux, I'm solving the problem of cross compiling, by
trying to eliminate anyone else's need to do it (at least as part of
Linux development).

The solution is in two parts: substitute native Linux development
environments running under emulation, and perform the cross compiling
necessary to _get_ to that native development environment in a
reproducible way for a lot of different plaforms (so you're not trusting
my magic binaries but don't have to do it yourself either).

In the process I'm answering some related questions: "how small and
simple can I get a native development environment" (to reduce the bits
that need to be cross compiled), how do I get the utterly sucky
performance of emulators to a point people are willing to put up with it
(pick a good emulator and install distcc to help out), and "how do you
bootstrap up from there to more complex native build environments in a
target-agnostic way" (all those build-images as examples).

But the core idea here is "get rid of the need for cross compiling", and
that lets me know where my project STOPS.

It lets me know I don't care about drivers for specific boards, that I
don't care about QT, that I don't care about package management on the
target, or target code installation (bootloaders, jtags, sdcard
partitioning)... These are all things where once people have a working
native development environment, they can take it from there. I don't
_have_ to solve that for them.

Your project seems to have a less clear dividing line between what you
do and what you DON'T do. If you can't say "this is not part of this
project", you get feature creep...

> Maintaining stability for various aspects such as the drivers mentioned
> above can be a pain.  While there will have to be testing and such for
> third party hardware, the business will release "official" hardware that
> will obviously be tested with the XiniX distro so that these issues
> won't be a factor as often.

Do you have any idea how _much_ third party hardware there is out there?

>     > 4. A better, more logical division of the system and user for various
>     > reasons.
> 
>     Um, like Apple did with macosx? Like that /usr->/usr/bin move? Like
>     gobolinux where they just went "screw it" and moved everything for no
>     apparent reason?
> 
> 
> Yeah something like that. :)  For example XiniX actually doesn't have a
> /usr directory at all, but instead software is considered "system"
> (which will be added to the firmware /bin, /sbin), "personal" (~/.bin,
> ~/.sbin), or "community" (/home/public/.bin, /home/public/.sbin).  This
> is done, again, to separate the system from the user/userbase and is
> necessary for some of the design goals.  There are a few other
> non-traditional aspects of the system too.

Apple got away with it because they're apple. Fedora is trying to make a
very minimal change in a backwards compatible way. You're adding
_hidden_ executable directories... I really don't see that ending well.

>     > While those are just a handful of ideas behind the distro, I'm not
>     sure
>     > if that's what you were looking for.  Currently the distro (named
>     XiniX
>     > - pronounced "zen ics") uses busybox, but I'd love to replace it with
>     > toybox once the required binaries are incorporated.
> 
>     I'm all for it...
> 
> 
> Great!  I'll keep my eye on the development and incorporate it as soon
> as I can!

I'm working on it. Got a bit mugged by real life again...

>     That's pretty much where I stop. I provide a basic tool, what people do
>     with it from there I'm intentionally agnostic about because there are a
>     more use cases for a knife than for nail clippers and vegetable peelers
>     combined.
> 
> That simple system would be ideal for a VM server though, don't you
> think?  After all, that system should *not* contain anything but the
> necessities to boot and run whatever, more complex systems via VM's right?

Sure. I'm always happy when people find a use for the tools I provide,
but I'm trying hard to stay agnostic about what use those tools might be
put to.

I'll happily respond to "I need it to do X because Y", that's great
feedback. But "my hammer keeps scratching the surfaces I'm pounding
dents out of" means I say "here's how you can wrap some cloth around it,
and some example cloth", not "from now on it will come wrapped in cloth
by default".

> So are you trying to accomplish the goal of developing an SDK with
> aboriginal Linux?  What are your goals and intended audience with that
> distro?

See above. "Make cross compiling go away."

Note that cross compiling includes "building a uClibc system under a
glibc system", and next up a musl system. (Assembling a toolchain in
general is hard and fiddly.)

>     Rob
>     --
>     GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
>     Either it's "mere aggregation", or a license violation.  Pick one.
> 
> 
> Looking at your signature, it reminded me of another questions I wanted
> to ask you.  If I were to guess, I'd say that you were more of a GPLv2
> guy.  Having said that, and I know this could take up pages worth of
> information, but as an overview why the dislike for the GPLv3?  Perhaps
> 'dislike' isn't the right word, but I'm sure you understand what I'm asking.

I need to finish writing up my thing on that. I've got a long file full
of notes...

> As always, thanks for your time!
> Dave

Sorry I'm so far behind. I need to amend the kernel MAINTAINERS file to
exclude the device trees subdirectory, but I've been holding off until I
get the PGP signatures to reestablish kernel.org access so I can put a
git tree up there I can submit stuff through, and sometimes "I refuse to
fix this until I do it RIGHT" can lead to piles of side effects imposed
upon oneself as a weird sort of peanance. My inbox is like that right now.

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.