From rob at landley.net  Sat Jul 21 05:05:52 2012
From: rob at landley.net (Rob Landley)
Date: Sat, 21 Jul 2012 00:05:52 -0500
Subject: [Aboriginal] Fwd: Re: greetings!
In-Reply-To: <50076C48.5010403@landley.net>
References: <50076C48.5010403@landley.net>
Message-ID: <500A38B0.9030301@landley.net>

And the next bit. Caught up now, I think.

-------- Original Message --------
Subject: Re: greetings!
Date: Wed, 18 Jul 2012 21:09:12 -0500
From: Rob Landley <rob at landley.net>
To: David Henderson <dhenderson at digital-pipe.com>

On 07/18/2012 08:13 AM, David Henderson wrote:
> Just following-up, I know you're a busy person. :)
> 
> Dave

Do you mind if I copy this to the aboriginal mailing list? People keep
emailing me off-list and I really should cc: the list to keep the
cobwebs and tumbleweeds down...

> On Thu, Jul 12, 2012 at 10:24 AM, David Henderson
> <dhenderson at digital-pipe.com <mailto:dhenderson at digital-pipe.com>> wrote:
> 
>     Wow and I often complain about my schedule! :)  Seems like you're
>     working on 28 hour work days! lol  This might be a good opportunity
>     for you to possibly offload some work then - at least on the distro
>     creation/maintenance side.

I'd love to. Did you ever read the presentation I have linked from the
left edge of http://landley.net/aboriginal ? The "html" version is
probably the most easily legible. (The PDF is the same but it's 90 megs
thanks to Apple impress being inefficient. The raw text one is harder to
follow, It think.)

It's a bit stale, but describes the design goals pretty well. Not just
what I did, but more importantly _why_.

>     My replies are in-line below to prevent confusion...
...
>     Their "work" would be anything the user normally does during the
>     day: email, word processing, listen to music, watch videos, program,
>     etc.

That's a full distro. As the presentation I mentioned covers, there are
a lot of distros out there already, which maintain enormous package
repositories with dependency information and build recipes and install
information and all that, and it's a full-time team of a dozen or more
people for each distro. Fedora and SuSE and Mandrivia do RPM, Debian and
Ubuntu do dpkg, Gentoo does Portage, Slackware does its tarballs, arch
linux has pacman, openembeded has its bitbake recipes, and so on.

Buildroot fell into the trap of doing that, and it pretty quickly took
up all their time. I made the explicit decision _not_ to do that. I
produced a native build environment on target and then _stopped_.

Separately, there's the control images:

  http://landley.net/hg/control-images
  http://landley.net/aboriginal/control-images

But that's designed to be as orthogonal as possible, I just haven't had
_time_ for that recently. (You'll notice the automated linux from
scratch build is a couple versions behind already, 6.8 I think.)

What I _wanted_ to do was natively bootstrap distros under the target
build environment, but it turns out that Fedora assumes it's building on
Fedora, and Debian assumes it's building on Debian, and even Gentoo
adamantly _refuses_ to build under anything but gentoo and how you
bootstrap from a bare environment is undocumented black magic you have
to reverse engineer. I was doing that, but wandered off to do toybox,
and until toybox goes 1.0 I'm unlikely to have time to work on control
images. I'm not really properly keeping up with kernel, busybox, and
uClibc package releases in the base system...

>  While the distro won't contain all of the software for
>     everyones "work" (e.g. CAD users), it'll start with a set of
>     software to perform the most common tasks.  As time elapses, more
>     and more can be added to include users with more rare "work".  This
>     software, however, will be installed with their "personal data" as
>     mentioned below.

I found it to be a can of worms.  Read the presentation for details.

>     Wow thats pretty interesting!  Currently XiniX uses grub, which I
>     don't think has an option like that.  Also, XiniX has the "firmware"
>     and Kernel separated (I didn't even know those two could be
>     combined).  I suppose the integration of the two can help maintain
>     system stability since everything is tested and released
>     together...  Did you ever run into any negative aspects when working
>     like this?  The only one I can see off the top of my head is that if
>     the distro isn't updated routinely, the kernel will get dated and
>     present exploitable bugs to crackers.

You can have the same problem with userspace.

It was just a packaging method. I could do it with the current stuff,
It's just qemu renders it less interesting, so I ship tarballs instead.

But that's what got me into initramfs, and why I wrote the initramfs
documentation (because I was an early adopter in the days you had to
read the code and ask the developers to see how to get it to work). I
needed some userspace code to run when the kernel came up to find and
mount the root filesystem and switch_root into it. (It's also why I
wrote a switch_root that wound up in busybox.)

>     Currently XiniX is CLI only, but hopefully by the end of next month
>     the graphical side will be up and running.

I got a gui running on the Qualcomm Hexagon natively built under
aboriginal by following the beyond linux from scratch instructions in
2010. (Alas, the scripts I wrote to do it were written on company time,
and stayed with Qualcomm. I haven't gotten around to recreating open
source versions yet...)

>  In order to keep the
>     size down while maintaining flexibility, the GUI is entirely
>     web-based so it works with head and headless devices in a uniform,
>     consistent way.  Also it does have software such as QT and/or GTK
>     installed for software that currently doesn't have a
>     web-interface-only alternative (and is also required for the webkit
>     backend).

I keep meaning to test more stuff out with uClibc++.  Especially since
that's being actively developeda gain...

But I also mean to switch gradually over from uClibc to musl...

>     Maintaining stability for various aspects such as the drivers
>     mentioned above can be a pain.  While there will have to be testing
>     and such for third party hardware, the business will release
>     "official" hardware that will obviously be tested with the XiniX
>     distro so that these issues won't be a factor as often.

I keep reading that as "Xilinx" and wondering how fast microblaze is
these days...

>         > 4. A better, more logical division of the system and user for
>         various
>         > reasons.
> 
>         Um, like Apple did with macosx? Like that /usr->/usr/bin move? Like
>         gobolinux where they just went "screw it" and moved everything
>         for no
>         apparent reason?
> 
> 
>     Yeah something like that. :)  For example XiniX actually doesn't
>     have a /usr directory at all, but instead software is considered
>     "system" (which will be added to the firmware /bin, /sbin),
>     "personal" (~/.bin, ~/.sbin), or "community" (/home/public/.bin,
>     /home/public/.sbin).  This is done, again, to separate the system
>     from the user/userbase and is necessary for some of the design
>     goals.  There are a few other non-traditional aspects of the system too.

Sigh.

  http://landley.net/writing/hackermonthly-issue022-pg33.pdf

There's a corollary to Moore's Law, which applies it to software. It
says that 50% of what you know is obsolete every 18 months.

I have a corollary to that corollary, which says "The great thing about
Unix is it's the same 50% cycling out over and over." This is how it
gradually builds a userbase, because you can wander away from it for 5,
7, even 10 years and come back and still find your way around the basic
stuff (posix-2001 still more or less applies), whereas over in
windows-land they obsoleted Visual Basic and replaced it with .net, and
then obsoleted .net (whatever that was, they were never quite clear) in
favor of programming windows 8 in HTML5...

[google how microsoft lost the API war, by joel spolsky]

>         That's pretty much where I stop. I provide a basic tool, what
>         people do
>         with it from there I'm intentionally agnostic about because
>         there are a
>         more use cases for a knife than for nail clippers and vegetable
>         peelers
>         combined.
> 
> 
>     That simple system would be ideal for a VM server though, don't you
>     think?  After all, that system should *not* contain anything but the
>     necessities to boot and run whatever, more complex systems via VM's
>     right?

The idea behind the control images is you run one within a system-image
for your target, it copies the read only filesystem into a chroot,
builds and installs packages into it, and when it's done tars up the
result and uploads the tarball to the host.

So you can automate any delta you like on top of the base system: I
don't have to care.

Probably I should document the control image stuff more thoroughly...

>     So are you trying to accomplish the goal of developing an SDK with
>     aboriginal Linux?  What are your goals and intended audience with
>     that distro?

It was in the presentation.html, but I should probably write up
something more _concise_.

I keep trying to do about.html but it's like trying to describe the uses
for a knife. It cuts, it spreads, you can whittle, it can be used for
self defense, you can throw it, you can clean your nails with it, it's a
basic flexible tool with minimal assumptions about its eventual use cases.

It's really easy to describe what a vegetable peeler is for. It peels
vegetables. That's about it.

>         Rob
>         --
>         GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
>         Either it's "mere aggregation", or a license violation.  Pick one.
> 
> 
>     Looking at your signature, it reminded me of another questions I
>     wanted to ask you.  If I were to guess, I'd say that you were more
>     of a GPLv2 guy.  Having said that, and I know this could take up
>     pages worth of information, but as an overview why the dislike for
>     the GPLv3?  Perhaps 'dislike' isn't the right word, but I'm sure you
>     understand what I'm asking.

Heh.

Lemme do a separate email for that one.

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.

 1342847152.0

From rob at landley.net  Sat Jul 21 05:04:25 2012
From: rob at landley.net (Rob Landley)
Date: Sat, 21 Jul 2012 00:04:25 -0500
Subject: [Aboriginal] Fwd: Re: greetings!
In-Reply-To: <4FFDACF5.9050108@landley.net>
References: <4FFDACF5.9050108@landley.net>
Message-ID: <500A3859.4090608@landley.net>

One of the various relevant conversations taking place offline...

-------- Original Message --------
Subject: Re: greetings!
Date: Wed, 11 Jul 2012 11:42:29 -0500
From: Rob Landley <rob at landley.net>
To: David Henderson <dhenderson at digital-pipe.com>

On 07/10/2012 01:40 PM, David Henderson wrote:
> Desperately short of free time - you're not preparing to move again are
> you?! :)

No, just full-time day job (rather than a contracting thing that ate my
life for 3-9 months and then gave me a few months off to do full-time
open source work; I've only had one month off in the past year and that
was switching from .)

Plus toybox (one person tackling the busybox problem space, implementing
fresh versions of the posix-2008, LSB 4.1, and android commands). And
Aboriginal Linux still needs maintenance (package upgrades break stuff,
distro upgrades break stuff), and could use some more platforms added
(m68k?), and I'd still like to get distro bootstrapping implemented
someday...

Oh, and I became the linux-kernel Documentation maintainer, which is
seriously filling up my inbox.

>  Anyhow... you left me with a pretty open-ended question ("What
> kind of linux distro), so I'll respond by saying:
> 
> 1. A very small distro acting almost like a firmware for the device
> (desktop, laptop, router, DVR, smart phone, etc) just to get the user to
> the point where (s)he can focus on their work.  After all, do one thing
> and do it well!

Define "their work"?

I did "boot to a shell prompt on serial console, with native development
tools", targeting various qemu platforms. This let people build code
natively.

I didn't really tackle "what tools do you need on target", because
everybody's needs are different and if you can build it natively, you've
got wget and a toolchain.

The proliferation of boards is kind of amazing. A half-dozen projects
from buildroot to OpenEmbedded accumulated buckets of board
configurations, but never cleanly separated them out from toolchain and
root filesystem config, and I decided the correct thing to do was get
defconfigs upstream into linus's git.

> 2. Although the OS can be installed and run from a storage device, it's
> intended to take a minimal amount of memory as it's home - RAM based. 
> This should make upgrading as simple as replacing a single file!

I remember ten years ago working out how to combine a kernel and root
filesystem into a single file, and patching lilo to have a "length"
parameter so it would only load the kernel and not the whole root
filesystem into memory.

  http://landley.net/aboriginal/old

I even worked out some unused space in the ELF header (for user mode
linux, the emulator I was using at the time) which was the old floppy
boot sector on real kernels so either way "unused space" that I could
stamp a 4 byte length to say what offset losetup should use creating the
loopback file for the root filesystem. (I used zisofs because squashfs
wasn't merged yet.) Getting an initrd built into the kernel so it could
do the loopback setup and hand off is what got me poking at initramfs in
the first place...

Good times, good times...

> 3. Having an easily usable interface that's common among the various
> devices while being as simplistic and efficient as possible.

Shell prompt?  xfce?  Web gui for headless boxes?

Just getting the _drivers_ right for a wide variety of boards was a huge
pain for me, and I was focusing on qemu. And I have to regression test
them every kernel release and bisect some really nasty stuff sometimes.
(I'm doing a writeup the most recent round of git bisect to teach people
how to bisect bugs under adverse conditions. The one that broke armv5l's
scsi driver was buried several bugs deep behind "the serial console
broke", "build break", and so on...)

> 4. A better, more logical division of the system and user for various
> reasons.

Um, like Apple did with macosx? Like that /usr->/usr/bin move? Like
gobolinux where they just went "screw it" and moved everything for no
apparent reason?

> While those are just a handful of ideas behind the distro, I'm not sure
> if that's what you were looking for.  Currently the distro (named XiniX
> - pronounced "zen ics") uses busybox, but I'd love to replace it with
> toybox once the required binaries are incorporated.

I'm all for it...

> I'd also like to
> say that looking over some of the ideas behind your aboriginal Linux, I
> can easily see it as a server edition used for VM's for a small/medium
> sized organization (or even an SDK via VM's).

I started by trying to provide the simplest system that can rebuild
itself under itself. Over time, that expanded to being able to cross
compile it to an arbitrary target and run it as a build environment
under an emulator.

That's pretty much where I stop. I provide a basic tool, what people do
with it from there I'm intentionally agnostic about because there are a
more use cases for a knife than for nail clippers and vegetable peelers
combined.

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.

 1342847065.0

From dhenderson at digital-pipe.com  Mon Jul 23 12:41:02 2012
From: dhenderson at digital-pipe.com (David Henderson)
Date: Mon, 23 Jul 2012 08:41:02 -0400
Subject: [Aboriginal] Fwd: greetings!
In-Reply-To: <CAF-YAQycVfooRTrgvx0sdJ0L9SxBRXA0WOVobGH_Qd1bW=3Buw@mail.gmail.com>
References: <CAF-YAQwqv5jjkS0AO=hfyySH7X1k2NA_8YF5FT__M4kSLAq2BQ@mail.gmail.com>
	<4FFB9E95.9050302@landley.net>
	<CAF-YAQww-HvJ-Kb4WL=BBwgAUvtGGu6u1bt2wmHHSFMN4-SS_A@mail.gmail.com>
	<4FFDACF5.9050108@landley.net>
	<CAF-YAQxkJ=uMMAfT+SAHizsNeHLR1FUd14gP+Zwv69anZC1nkg@mail.gmail.com>
	<CAF-YAQysiaOmmRcG=Yc1qYUcdY=Q4LMBKCEhRfXb4ChtVEv61g@mail.gmail.com>
	<50076C48.5010403@landley.net>
	<CAF-YAQycVfooRTrgvx0sdJ0L9SxBRXA0WOVobGH_Qd1bW=3Buw@mail.gmail.com>
Message-ID: <CAF-YAQzh16eFLTB=qO4876L3WsrZCVia1aQM-JV_jiZza=ZqfA@mail.gmail.com>

On Wed, Jul 18, 2012 at 10:09 PM, Rob Landley <rob at landley.net> wrote:

> On 07/18/2012 08:13 AM, David Henderson wrote:
> > Just following-up, I know you're a busy person. :)
> >
> > Dave
>
> Do you mind if I copy this to the aboriginal mailing list? People keep
> emailing me off-list and I really should cc: the list to keep the
> cobwebs and tumbleweeds down...
>

Sure!  It never hurts to hear what others have to say right? :)


>
> > On Thu, Jul 12, 2012 at 10:24 AM, David Henderson
> > <dhenderson at digital-pipe.com <mailto:dhenderson at digital-pipe.com>>
> wrote:
> >
> >     Wow and I often complain about my schedule! :)  Seems like you're
> >     working on 28 hour work days! lol  This might be a good opportunity
> >     for you to possibly offload some work then - at least on the distro
> >     creation/maintenance side.
>
> I'd love to. Did you ever read the presentation I have linked from the
> left edge of http://landley.net/aboriginal ? The "html" version is
> probably the most easily legible. (The PDF is the same but it's 90 megs
> thanks to Apple impress being inefficient. The raw text one is harder to
> follow, It think.)
>
> It's a bit stale, but describes the design goals pretty well. Not just
> what I did, but more importantly _why_.
>

No I didn't, but did started to early this morning.  I'll have to come back
to it however, as I was more interested in the other referenced material
below.  I'll see about completing it over the weekend.


>
> >     My replies are in-line below to prevent confusion...
> ...
> >     Their "work" would be anything the user normally does during the
> >     day: email, word processing, listen to music, watch videos, program,
> >     etc.
>
> That's a full distro. As the presentation I mentioned covers, there are
> a lot of distros out there already, which maintain enormous package
> repositories with dependency information and build recipes and install
> information and all that, and it's a full-time team of a dozen or more
> people for each distro.


Combined, you're right that is a full distro, however, XiniX is just the
"firmware" to get the user to the point where (s)he can do the "work", but
doesn't include the software to actually do the "work".  Basically our goal
will be to create XiniX and a base set of weblications to perform basic
tasks - surfing the Internet, communication (email, IM, IRC, etc), and
office tasks (Word Processing, Spreadsheets, etc) as well as some common
"at home" tasks like watching movies, playing music, etc.  Programs outside
of our "official" umbrella will be left to the community to add to (and
maintain) the software repositories.


> Fedora and SuSE and Mandrivia do RPM, Debian and
> Ubuntu do dpkg, Gentoo does Portage, Slackware does its tarballs, arch
> linux has pacman, openembeded has its bitbake recipes, and so on.
>
> Buildroot fell into the trap of doing that, and it pretty quickly took
> up all their time. I made the explicit decision _not_ to do that. I
> produced a native build environment on target and then _stopped_.
>
> Separately, there's the control images:
>
>   http://landley.net/hg/control-images
>   http://landley.net/aboriginal/control-images
>
> But that's designed to be as orthogonal as possible, I just haven't had
> _time_ for that recently. (You'll notice the automated linux from
> scratch build is a couple versions behind already, 6.8 I think.)
>
> What I _wanted_ to do was natively bootstrap distros under the target
> build environment, but it turns out that Fedora assumes it's building on
> Fedora, and Debian assumes it's building on Debian, and even Gentoo
> adamantly _refuses_ to build under anything but gentoo and how you
> bootstrap from a bare environment is undocumented black magic you have
> to reverse engineer. I was doing that, but wandered off to do toybox,
> and until toybox goes 1.0 I'm unlikely to have time to work on control
> images. I'm not really properly keeping up with kernel, busybox, and
> uClibc package releases in the base system...
>
> >  While the distro won't contain all of the software for
> >     everyones "work" (e.g. CAD users), it'll start with a set of
> >     software to perform the most common tasks.  As time elapses, more
> >     and more can be added to include users with more rare "work".  This
> >     software, however, will be installed with their "personal data" as
> >     mentioned below.
>
> I found it to be a can of worms.  Read the presentation for details.
>

Which aspect - adding software "on demand" or combining software as part of
the users "personal data"?


>
> >     Wow thats pretty interesting!  Currently XiniX uses grub, which I
> >     don't think has an option like that.  Also, XiniX has the "firmware"
> >     and Kernel separated (I didn't even know those two could be
> >     combined).  I suppose the integration of the two can help maintain
> >     system stability since everything is tested and released
> >     together...  Did you ever run into any negative aspects when working
> >     like this?  The only one I can see off the top of my head is that if
> >     the distro isn't updated routinely, the kernel will get dated and
> >     present exploitable bugs to crackers.
>
> You can have the same problem with userspace.
>
> It was just a packaging method. I could do it with the current stuff,
> It's just qemu renders it less interesting, so I ship tarballs instead.
>
> But that's what got me into initramfs, and why I wrote the initramfs
> documentation (because I was an early adopter in the days you had to
> read the code and ask the developers to see how to get it to work). I
> needed some userspace code to run when the kernel came up to find and
> mount the root filesystem and switch_root into it. (It's also why I
> wrote a switch_root that wound up in busybox.)
>
> >     Currently XiniX is CLI only, but hopefully by the end of next month
> >     the graphical side will be up and running.
>
> I got a gui running on the Qualcomm Hexagon natively built under
> aboriginal by following the beyond linux from scratch instructions in
> 2010. (Alas, the scripts I wrote to do it were written on company time,
> and stayed with Qualcomm. I haven't gotten around to recreating open
> source versions yet...)
>

What were you using as the graphics backend?  X11, DirectFB, etc?


>
> >  In order to keep the
> >     size down while maintaining flexibility, the GUI is entirely
> >     web-based so it works with head and headless devices in a uniform,
> >     consistent way.  Also it does have software such as QT and/or GTK
> >     installed for software that currently doesn't have a
> >     web-interface-only alternative (and is also required for the webkit
> >     backend).
>
> I keep meaning to test more stuff out with uClibc++.  Especially since
> that's being actively developeda gain...
>
> But I also mean to switch gradually over from uClibc to musl...
>

Looking over the comparison chart at
http://www.etalabs.net/compare_libcs.html, I can see that it looks like the
developer of musl shares your design and coding philosophies.  How long
will the transition to musl take?


>
> >     Maintaining stability for various aspects such as the drivers
> >     mentioned above can be a pain.  While there will have to be testing
> >     and such for third party hardware, the business will release
> >     "official" hardware that will obviously be tested with the XiniX
> >     distro so that these issues won't be a factor as often.
>
> I keep reading that as "Xilinx" and wondering how fast microblaze is
> these days...
>
> >         > 4. A better, more logical division of the system and user for
> >         various
> >         > reasons.
> >
> >         Um, like Apple did with macosx? Like that /usr->/usr/bin move?
> Like
> >         gobolinux where they just went "screw it" and moved everything
> >         for no
> >         apparent reason?
> >
> >
> >     Yeah something like that. :)  For example XiniX actually doesn't
> >     have a /usr directory at all, but instead software is considered
> >     "system" (which will be added to the firmware /bin, /sbin),
> >     "personal" (~/.bin, ~/.sbin), or "community" (/home/public/.bin,
> >     /home/public/.sbin).  This is done, again, to separate the system
> >     from the user/userbase and is necessary for some of the design
> >     goals.  There are a few other non-traditional aspects of the system
> too.
>
> Sigh.
>
>   http://landley.net/writing/hackermonthly-issue022-pg33.pdf
>

Great article!  I actually wondered where all those directories came from.
lol  I know it sounds like I'm no better than the others in the article
that re-arrange the FS layout, but instead of having all these symlinked
directories and crazy list of mounts, wouldn't it make more sense to just
provide a common structure between the various aspects of the system?  To
me, it also provides an easy way to store all user data (personal and
community) under a single mount (/home).  For example:

* FIRMWARE (prefix is /) *
bin - stores publicly accessible binaries like mount, grep, tar, etc
etc - stores firmware configurations
lib - stores firmware shared libraries
sbin - stores administrator/system binaries mkfs, hdparam, ifconfig, etc
var - stores data like firmware logs, pid files, etc

* PERSONAL (prefix is /home/$USER) *
.bin - stores software specific for the user (e.g. pidgin, chatzilla, etc)
.etc - stores personal configuration for personal AND community software
.lib - stores shared libraries for personal software
.var - stores logs, pid files, etc for personal AND community software
NOTE: there's no .sbin since there really shouldn't be unusable binaries
for personally (or community) installed software

* COMMUNITY (prefix is /home/Public) *
.bin - stores software for all the users of the device so that one instance
of software isn't installed and maintained per user (e.g. OpenOffice.org),
plus it can save space
.etc - stores general configuration for community software (e.g. firefox)
.lib - stores shares libraries for community software
.var - stores logs and other info for community software (e.g. mplayer,
firefox, etc)
NOTE: there's no .sbin since there really shouldn't be unusable binaries
for personally (or community) installed software


>
> There's a corollary to Moore's Law, which applies it to software. It
> says that 50% of what you know is obsolete every 18 months.
>
> I have a corollary to that corollary, which says "The great thing about
> Unix is it's the same 50% cycling out over and over." This is how it
> gradually builds a userbase, because you can wander away from it for 5,
> 7, even 10 years and come back and still find your way around the basic
> stuff (posix-2001 still more or less applies), whereas over in
> windows-land they obsoleted Visual Basic and replaced it with .net, and
> then obsoleted .net (whatever that was, they were never quite clear) in
> favor of programming windows 8 in HTML5...
>
> [google how microsoft lost the API war, by joel spolsky]
>

Joel's article was absolutely fascinating!  Thanks for sharing it!  I've
actually been telling people about the transition into the webosphere for
many years now.  That's actually why I see XiniX being valuable to people
and organizations!


>
> >         That's pretty much where I stop. I provide a basic tool, what
> >         people do
> >         with it from there I'm intentionally agnostic about because
> >         there are a
> >         more use cases for a knife than for nail clippers and vegetable
> >         peelers
> >         combined.
> >
> >
> >     That simple system would be ideal for a VM server though, don't you
> >     think?  After all, that system should *not* contain anything but the
> >     necessities to boot and run whatever, more complex systems via VM's
> >     right?
>
> The idea behind the control images is you run one within a system-image
> for your target, it copies the read only filesystem into a chroot,
> builds and installs packages into it, and when it's done tars up the
> result and uploads the tarball to the host.
>
> So you can automate any delta you like on top of the base system: I
> don't have to care.
>
> Probably I should document the control image stuff more thoroughly...
>
> >     So are you trying to accomplish the goal of developing an SDK with
> >     aboriginal Linux?  What are your goals and intended audience with
> >     that distro?
>
> It was in the presentation.html, but I should probably write up
> something more _concise_.
>
> I keep trying to do about.html but it's like trying to describe the uses
> for a knife. It cuts, it spreads, you can whittle, it can be used for
> self defense, you can throw it, you can clean your nails with it, it's a
> basic flexible tool with minimal assumptions about its eventual use cases.
>
> It's really easy to describe what a vegetable peeler is for. It peels
> vegetables. That's about it.
>
> >         Rob
> >         --
> >         GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
> >         Either it's "mere aggregation", or a license violation.  Pick
> one.
> >
> >
> >     Looking at your signature, it reminded me of another questions I
> >     wanted to ask you.  If I were to guess, I'd say that you were more
> >     of a GPLv2 guy.  Having said that, and I know this could take up
> >     pages worth of information, but as an overview why the dislike for
> >     the GPLv3?  Perhaps 'dislike' isn't the right word, but I'm sure you
> >     understand what I'm asking.
>
> Heh.
>
> Lemme do a separate email for that one.
>
>
I'll be interested to read it! :)

Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20120723/8f7d2773/attachment.htm>

From rob at landley.net  Wed Jul 25 19:10:46 2012
From: rob at landley.net (Rob Landley)
Date: Wed, 25 Jul 2012 14:10:46 -0500
Subject: [Aboriginal] Fwd: greetings!
In-Reply-To: <CAF-YAQzh16eFLTB=qO4876L3WsrZCVia1aQM-JV_jiZza=ZqfA@mail.gmail.com>
References: <CAF-YAQwqv5jjkS0AO=hfyySH7X1k2NA_8YF5FT__M4kSLAq2BQ@mail.gmail.com>
	<4FFB9E95.9050302@landley.net>
	<CAF-YAQww-HvJ-Kb4WL=BBwgAUvtGGu6u1bt2wmHHSFMN4-SS_A@mail.gmail.com>
	<4FFDACF5.9050108@landley.net>
	<CAF-YAQxkJ=uMMAfT+SAHizsNeHLR1FUd14gP+Zwv69anZC1nkg@mail.gmail.com>
	<CAF-YAQysiaOmmRcG=Yc1qYUcdY=Q4LMBKCEhRfXb4ChtVEv61g@mail.gmail.com>
	<50076C48.5010403@landley.net>
	<CAF-YAQycVfooRTrgvx0sdJ0L9SxBRXA0WOVobGH_Qd1bW=3Buw@mail.gmail.com>
	<CAF-YAQzh16eFLTB=qO4876L3WsrZCVia1aQM-JV_jiZza=ZqfA@mail.gmail.com>
Message-ID: <501044B6.5010504@landley.net>

On 07/23/2012 07:41 AM, David Henderson wrote:
> On Wed, Jul 18, 2012 at 10:09 PM, Rob Landley <rob at landley.net
>     >     Their "work" would be anything the user normally does during the
>     >     day: email, word processing, listen to music, watch videos,
>     program,
>     >     etc.
> 
>     That's a full distro. As the presentation I mentioned covers, there are
>     a lot of distros out there already, which maintain enormous package
>     repositories with dependency information and build recipes and install
>     information and all that, and it's a full-time team of a dozen or more
>     people for each distro.
> 
> Combined, you're right that is a full distro, however, XiniX is just the
> "firmware" to get the user to the point where (s)he can do the "work",
> but doesn't include the software to actually do the "work".

That line turns out to be hard to draw, or at least I found it so.

> Basically
> our goal will be to create XiniX and a base set of weblications to
> perform basic tasks - surfing the Internet, communication (email, IM,
> IRC, etc), and office tasks (Word Processing, Spreadsheets, etc) as well
> as some common "at home" tasks like watching movies, playing music,
> etc.

There's an old problem in computer science: users only use 20% of your
app's features. But they can't agree on which 20%.

> Programs outside of our "official" umbrella will be left to the
> community to add to (and maintain) the software repositories.

There are literally hundreds of linux distributions trying to do this
already. Just this week I bumped into

>     >  While the distro won't contain all of the software for
>     >     everyones "work" (e.g. CAD users), it'll start with a set of
>     >     software to perform the most common tasks.  As time elapses, more
>     >     and more can be added to include users with more rare "work".
>      This
>     >     software, however, will be installed with their "personal data" as
>     >     mentioned below.
> 
>     I found it to be a can of worms.  Read the presentation for details.
> 
> Which aspect - adding software "on demand" or combining software as part
> of the users "personal data"?

Figuring out which software to include, and maintaining the build
recipes when some library decides to change its API again and two
different packages need different versions of glib and so on...

>     I got a gui running on the Qualcomm Hexagon natively built under
>     aboriginal by following the beyond linux from scratch instructions in
>     2010. (Alas, the scripts I wrote to do it were written on company time,
>     and stayed with Qualcomm. I haven't gotten around to recreating open
>     source versions yet...)
> 
> What were you using as the graphics backend?  X11, DirectFB, etc?

The board didn't have a graphics card so we exported the display to
other machines through the net. (We tried getting the vnc plugin of
x.org running but there was some problem, possibly not enough memory? I
forget. Might just have been not enough time.)

>     But I also mean to switch gradually over from uClibc to musl...
> 
> Looking over the comparison chart at
> http://www.etalabs.net/compare_libcs.html, I can see that it looks like
> the developer of musl shares your design and coding philosophies.  How
> long will the transition to musl take?

No idea, but the real limiting factor (other than my shortage of time)
is when ports to the architectures I want to support show up.

uClibc has been stagnant for half a decade. It's unblocked a bit
recently but it's still impossible to bisect a problem between two
releases (trivial problem: they branch-then-tag. Bigger problem:
interstitial versions don't even compile.)

>     >     Yeah something like that. :)  For example XiniX actually doesn't
>     >     have a /usr directory at all, but instead software is considered
>     >     "system" (which will be added to the firmware /bin, /sbin),
>     >     "personal" (~/.bin, ~/.sbin), or "community" (/home/public/.bin,
>     >     /home/public/.sbin).  This is done, again, to separate the system
>     >     from the user/userbase and is necessary for some of the design
>     >     goals.  There are a few other non-traditional aspects of the
>     system too.
> 
>     Sigh.
> 
>       http://landley.net/writing/hackermonthly-issue022-pg33.pdf
> 
> 
> Great article!  I actually wondered where all those directories came
> from. lol  I know it sounds like I'm no better than the others in the
> article that re-arrange the FS layout, but instead of having all these
> symlinked directories and crazy list of mounts, wouldn't it make more
> sense to just provide a common structure between the various aspects of
> the system?  To me, it also provides an easy way to store all user data
> (personal and community) under a single mount (/home).  For example:

The qwerty keyboard layout is universally acknowledged to suck.

Everybody can sit down at a qwerty keyboard and type on it.

The value _is_ that it's standardized. Not that the standard makes
sense. Thus the only way to get any sort of momentum behind an
alternative is to have somebody like Apple or Google behind it, and to
be consistent for about 5 years before anybody starts to care about the
new layout.

>     There's a corollary to Moore's Law, which applies it to software. It
>     says that 50% of what you know is obsolete every 18 months.
> 
>     I have a corollary to that corollary, which says "The great thing about
>     Unix is it's the same 50% cycling out over and over." This is how it
>     gradually builds a userbase, because you can wander away from it for 5,
>     7, even 10 years and come back and still find your way around the basic
>     stuff (posix-2001 still more or less applies), whereas over in
>     windows-land they obsoleted Visual Basic and replaced it with .net, and
>     then obsoleted .net (whatever that was, they were never quite clear) in
>     favor of programming windows 8 in HTML5...
> 
>     [google how microsoft lost the API war, by joel spolsky]
> 
> Joel's article was absolutely fascinating!  Thanks for sharing it!

The web not only has cool things on it now, it had cool things on it
before now. :)

> I've
> actually been telling people about the transition into the webosphere
> for many years now.  That's actually why I see XiniX being valuable to
> people and organizations!

The "transition to the webosphere" is what Google's Chromebooks were
trying to address, and they're still dinking at it:

http://www.h-online.com/open/news/item/Google-packages-web-apps-for-the-desktop-1650054.html

It's what html5 is targeting. HTML5 fragmentation is fun to watch:

http://www.h-online.com/open/news/item/WHATWG-and-W3C-diverge-over-HTML5-development-1649946.html
http://www.h-online.com/open/news/item/W3C-HTML5-standardisation-gets-financial-backing-1651829.html

Even Windows 8 is requiring everything to be html5...

>     >     Looking at your signature, it reminded me of another questions I
>     >     wanted to ask you.  If I were to guess, I'd say that you were more
>     >     of a GPLv2 guy.  Having said that, and I know this could take up
>     >     pages worth of information, but as an overview why the dislike for
>     >     the GPLv3?  Perhaps 'dislike' isn't the right word, but I'm
>     sure you
>     >     understand what I'm asking.
> 
>     Heh.
> 
>     Lemme do a separate email for that one.

I keep writing up _bits_ of it in various places.  I wrote up a couple
facets of "why busybox shouldn't move to gplv3 only" back in 2006:

  https://lwn.net/Articles/202110/
  http://landley.net/notes-2006.html#03-12-2006

The most recent two are here, talking about splitting the community and
undermining the network effects we'd benefitted from:

  http://lwn.net/Articles/504247/
  http://lwn.net/Articles/505644/

A quick search brought up a bunch of random blog entries that wing the
topic in passing.  I need to go through these too:

http://landley.net/notes-2006.html#14-09-2006
http://landley.net/notes-2006.html#18-09-2006
http://landley.net/notes-2006.html#20-09-2006
http://landley.net/notes-2006.html#23-09-2006
http://landley.net/notes-2006.html#26-09-2006
http;//landley.net/notes-2007.html#20-09-2007
http://landley.net/notes-2007.html#20-11-2007
http://landley.net/notes-2008.html#18-03-2008
http://landley.net/notes-2008.html#22-08-2008
http://landley.net/notes-2008.html#23-08-2008
http://landley.net/notes-2008.html#26-08-2008
http://landley.net/notes-2008.html#19-09-2008
http://landley.net/notes-2008.html#15-10-2008
http://landley.net/notes-2008.html#11-12-2008
http://landley.net/notes-2008.html#14-12-2008
http://landley.net/notes-2009.html#01-03-2009
http://landley.net/notes-2009.html#02-03-2009
http://landley.net/notes-2009.html#17-08-2009
http://landley.net/notes-2009.html#15-12-2009
http://landley.net/notes-2010.html#10-01-2010
http://landley.net/notes-2010.html#19-07-2010
http://landley.net/notes-2010.html#09-08-2010
http://landley.net/notes-2011.html#20-05-2011
http://landley.net/notes-2011.html#15-08-2011
http://landley.net/notes-2011.html#13-11-2011
http://landley.net/notes-2011.html#16-12-2011

I'm assembling a list of topics at
http://landley.net/toybox/oldlicense.txt that I should cover in a once
and for all writeup.

(Oh, and
http://archive.org/details/EbenMoglenLectureEdinburghJune2007StreamingVideo384kbits
is really good, even if he's on the other side.)

> I'll be interested to read it! :)

Above is the slush pile. (And not even the _entire_ slush pile.) I need
to digest that into something manageable.

> Dave
> 
> 
> 
> _______________________________________________
> Aboriginal mailing list
> Aboriginal at lists.landley.net
> http://lists.landley.net/listinfo.cgi/aboriginal-landley.net


-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.

 1343243446.0

From rob at landley.net  Thu Jul 26 00:37:34 2012
From: rob at landley.net (Rob Landley)
Date: Wed, 25 Jul 2012 19:37:34 -0500
Subject: [Aboriginal] I aten't dead.
Message-ID: <5010914E.3060102@landley.net>

It has been WAY TOO LONG since I cut a release.

The simple fact is toybox has been eating all my hobbyist development
time (which is maybe 10 hours a week, and most of that on weekends, due
to day job and the commute sucking away all my energy.) But I just cut a
toybox 0.4.0 release so this has swung back around to the top of my todo
list.

I integrated the 3.5 kernel (perl patch update, yank two now obsolete
patches), and it builds for most targets. (The repository also has 3.4
and 3.3 builds in there, I just didn't cut releases around them.)

i686 works, mips works, sparc works.

Todo list for cutting a release:

  arm versatile's scsi emulation broke AGAIN. Tracked to commit
  1bc39ac5dab but not immediately clear how to fix it. (They changed
  Interrupt routing to match documentation instead of reality, and
  the patch no longer cleanly reverts due ot something called
  "swizzle" having an infrastructure change...)

  sh4 is broken, not sure if this is qemu or kernel.

  powerpc build breaks in gcc, need to track that down. Not sure how
  the kernel did that, maybe it's because I'm testing with toybox.

  The x86-64 problem on Fedora is because libgcc_s.so is being built
  against uClibc and then LD_LIBRARY_PATH inserted into a shell script,
  and /bin/bash uses that library. It's plain and simple host/target
  confusion, what I haven't figured yet is why this works
  elsewhere. I'm torn between committing a quick hack workaround and
  digging in until I understand it properly.

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.

 1343263054.0

From dhenderson at digital-pipe.com  Thu Jul 26 13:17:14 2012
From: dhenderson at digital-pipe.com (David Henderson)
Date: Thu, 26 Jul 2012 09:17:14 -0400
Subject: [Aboriginal] Fwd: greetings!
In-Reply-To: <501044B6.5010504@landley.net>
References: <CAF-YAQwqv5jjkS0AO=hfyySH7X1k2NA_8YF5FT__M4kSLAq2BQ@mail.gmail.com>
	<4FFB9E95.9050302@landley.net>
	<CAF-YAQww-HvJ-Kb4WL=BBwgAUvtGGu6u1bt2wmHHSFMN4-SS_A@mail.gmail.com>
	<4FFDACF5.9050108@landley.net>
	<CAF-YAQxkJ=uMMAfT+SAHizsNeHLR1FUd14gP+Zwv69anZC1nkg@mail.gmail.com>
	<CAF-YAQysiaOmmRcG=Yc1qYUcdY=Q4LMBKCEhRfXb4ChtVEv61g@mail.gmail.com>
	<50076C48.5010403@landley.net>
	<CAF-YAQycVfooRTrgvx0sdJ0L9SxBRXA0WOVobGH_Qd1bW=3Buw@mail.gmail.com>
	<CAF-YAQzh16eFLTB=qO4876L3WsrZCVia1aQM-JV_jiZza=ZqfA@mail.gmail.com>
	<501044B6.5010504@landley.net>
Message-ID: <CAF-YAQwXWRNrZS3RHyTV8sCZtpDzxuMV37qMUFRZYF=ykzNdXA@mail.gmail.com>

This email thread has started to become soup!  So... I'll start a new
paragraph so we can start over. :)

Going back to the software included in the "firmware", the list should be
very small and only allow the device to get to the point for user
interaction.  For example,

- busybox (later toybox) for core functionality including user management,
scheduling, etc
- bash
- clAPI (our own software)
- the appropriate mount binaries for ext, ntfs, nfs, and smb/cifs
- ssh
- ssl (for secured headless devices or other communication)
- ssmtp (or similar) for automated outbound communication of device
* optional alsa (for sound enabled devices)
* optional graphics software for headed devices (headless will already be
handled via clAPI)

Items such as mplayer, samba, rdesktop, okular, perl, rsync, etc will be
outside the scope of the "firmware" because they aren't required to turn on
a device, login, and get to either the prompt or blank desktop (GUI).  The
list above should enable a device to communicate/interact with other
devices on the most basic level, but nothing beyond.
Because the "firmware" is kept to an absolute minimum, I think any library
API changes will not really disrupt the device from working.


Regarding the musl switch over, I had no clue that uClibc has been shelved
for so long!  Why wouldn't the switch to musl be more critical?  Not only
would you get actively developed software, but the speed increases as shown
in one of the links below.


I can see your point about standardization in the qwerty keyboard example
provided, however, because of how Linux has evolved compared to the design
goals of the XiniX distro, it's easier to manage the OS by separating it as
I have.  Otherwise, there would be an unwieldy amount of mounts to maintain
with software and configs going in various places (e.g. /bin, /usr/bin,
/usr/local/bin, /usr/share, etc).  The layout for XiniX is uniform and
makes sense where things are located and why.


Thanks for all the articles on the GPLv2 vs GPLv3 - lots of reading to do!
I'm sure I'll have more questions for you once I'm done. :)

Dave


On Wed, Jul 25, 2012 at 3:10 PM, Rob Landley <rob at landley.net> wrote:

> On 07/23/2012 07:41 AM, David Henderson wrote:
> > On Wed, Jul 18, 2012 at 10:09 PM, Rob Landley <rob at landley.net
> >     >     Their "work" would be anything the user normally does during
> the
> >     >     day: email, word processing, listen to music, watch videos,
> >     program,
> >     >     etc.
> >
> >     That's a full distro. As the presentation I mentioned covers, there
> are
> >     a lot of distros out there already, which maintain enormous package
> >     repositories with dependency information and build recipes and
> install
> >     information and all that, and it's a full-time team of a dozen or
> more
> >     people for each distro.
> >
> > Combined, you're right that is a full distro, however, XiniX is just the
> > "firmware" to get the user to the point where (s)he can do the "work",
> > but doesn't include the software to actually do the "work".
>
> That line turns out to be hard to draw, or at least I found it so.
>
> > Basically
> > our goal will be to create XiniX and a base set of weblications to
> > perform basic tasks - surfing the Internet, communication (email, IM,
> > IRC, etc), and office tasks (Word Processing, Spreadsheets, etc) as well
> > as some common "at home" tasks like watching movies, playing music,
> > etc.
>
> There's an old problem in computer science: users only use 20% of your
> app's features. But they can't agree on which 20%.
>
> > Programs outside of our "official" umbrella will be left to the
> > community to add to (and maintain) the software repositories.
>
> There are literally hundreds of linux distributions trying to do this
> already. Just this week I bumped into
>
> >     >  While the distro won't contain all of the software for
> >     >     everyones "work" (e.g. CAD users), it'll start with a set of
> >     >     software to perform the most common tasks.  As time elapses,
> more
> >     >     and more can be added to include users with more rare "work".
> >      This
> >     >     software, however, will be installed with their "personal
> data" as
> >     >     mentioned below.
> >
> >     I found it to be a can of worms.  Read the presentation for details.
> >
> > Which aspect - adding software "on demand" or combining software as part
> > of the users "personal data"?
>
> Figuring out which software to include, and maintaining the build
> recipes when some library decides to change its API again and two
> different packages need different versions of glib and so on...
>
> >     I got a gui running on the Qualcomm Hexagon natively built under
> >     aboriginal by following the beyond linux from scratch instructions in
> >     2010. (Alas, the scripts I wrote to do it were written on company
> time,
> >     and stayed with Qualcomm. I haven't gotten around to recreating open
> >     source versions yet...)
> >
> > What were you using as the graphics backend?  X11, DirectFB, etc?
>
> The board didn't have a graphics card so we exported the display to
> other machines through the net. (We tried getting the vnc plugin of
> x.org running but there was some problem, possibly not enough memory? I
> forget. Might just have been not enough time.)
>
> >     But I also mean to switch gradually over from uClibc to musl...
> >
> > Looking over the comparison chart at
> > http://www.etalabs.net/compare_libcs.html, I can see that it looks like
> > the developer of musl shares your design and coding philosophies.  How
> > long will the transition to musl take?
>
> No idea, but the real limiting factor (other than my shortage of time)
> is when ports to the architectures I want to support show up.
>
> uClibc has been stagnant for half a decade. It's unblocked a bit
> recently but it's still impossible to bisect a problem between two
> releases (trivial problem: they branch-then-tag. Bigger problem:
> interstitial versions don't even compile.)
>
> >     >     Yeah something like that. :)  For example XiniX actually
> doesn't
> >     >     have a /usr directory at all, but instead software is
> considered
> >     >     "system" (which will be added to the firmware /bin, /sbin),
> >     >     "personal" (~/.bin, ~/.sbin), or "community"
> (/home/public/.bin,
> >     >     /home/public/.sbin).  This is done, again, to separate the
> system
> >     >     from the user/userbase and is necessary for some of the design
> >     >     goals.  There are a few other non-traditional aspects of the
> >     system too.
> >
> >     Sigh.
> >
> >       http://landley.net/writing/hackermonthly-issue022-pg33.pdf
> >
> >
> > Great article!  I actually wondered where all those directories came
> > from. lol  I know it sounds like I'm no better than the others in the
> > article that re-arrange the FS layout, but instead of having all these
> > symlinked directories and crazy list of mounts, wouldn't it make more
> > sense to just provide a common structure between the various aspects of
> > the system?  To me, it also provides an easy way to store all user data
> > (personal and community) under a single mount (/home).  For example:
>
> The qwerty keyboard layout is universally acknowledged to suck.
>
> Everybody can sit down at a qwerty keyboard and type on it.
>
> The value _is_ that it's standardized. Not that the standard makes
> sense. Thus the only way to get any sort of momentum behind an
> alternative is to have somebody like Apple or Google behind it, and to
> be consistent for about 5 years before anybody starts to care about the
> new layout.
>
> >     There's a corollary to Moore's Law, which applies it to software. It
> >     says that 50% of what you know is obsolete every 18 months.
> >
> >     I have a corollary to that corollary, which says "The great thing
> about
> >     Unix is it's the same 50% cycling out over and over." This is how it
> >     gradually builds a userbase, because you can wander away from it for
> 5,
> >     7, even 10 years and come back and still find your way around the
> basic
> >     stuff (posix-2001 still more or less applies), whereas over in
> >     windows-land they obsoleted Visual Basic and replaced it with .net,
> and
> >     then obsoleted .net (whatever that was, they were never quite clear)
> in
> >     favor of programming windows 8 in HTML5...
> >
> >     [google how microsoft lost the API war, by joel spolsky]
> >
> > Joel's article was absolutely fascinating!  Thanks for sharing it!
>
> The web not only has cool things on it now, it had cool things on it
> before now. :)
>
> > I've
> > actually been telling people about the transition into the webosphere
> > for many years now.  That's actually why I see XiniX being valuable to
> > people and organizations!
>
> The "transition to the webosphere" is what Google's Chromebooks were
> trying to address, and they're still dinking at it:
>
>
> http://www.h-online.com/open/news/item/Google-packages-web-apps-for-the-desktop-1650054.html
>
> It's what html5 is targeting. HTML5 fragmentation is fun to watch:
>
>
> http://www.h-online.com/open/news/item/WHATWG-and-W3C-diverge-over-HTML5-development-1649946.html
>
> http://www.h-online.com/open/news/item/W3C-HTML5-standardisation-gets-financial-backing-1651829.html
>
> Even Windows 8 is requiring everything to be html5...
>
> >     >     Looking at your signature, it reminded me of another questions
> I
> >     >     wanted to ask you.  If I were to guess, I'd say that you were
> more
> >     >     of a GPLv2 guy.  Having said that, and I know this could take
> up
> >     >     pages worth of information, but as an overview why the dislike
> for
> >     >     the GPLv3?  Perhaps 'dislike' isn't the right word, but I'm
> >     sure you
> >     >     understand what I'm asking.
> >
> >     Heh.
> >
> >     Lemme do a separate email for that one.
>
> I keep writing up _bits_ of it in various places.  I wrote up a couple
> facets of "why busybox shouldn't move to gplv3 only" back in 2006:
>
>   https://lwn.net/Articles/202110/
>   http://landley.net/notes-2006.html#03-12-2006
>
> The most recent two are here, talking about splitting the community and
> undermining the network effects we'd benefitted from:
>
>   http://lwn.net/Articles/504247/
>   http://lwn.net/Articles/505644/
>
> A quick search brought up a bunch of random blog entries that wing the
> topic in passing.  I need to go through these too:
>
> http://landley.net/notes-2006.html#14-09-2006
> http://landley.net/notes-2006.html#18-09-2006
> http://landley.net/notes-2006.html#20-09-2006
> http://landley.net/notes-2006.html#23-09-2006
> http://landley.net/notes-2006.html#26-09-2006
> http;//landley.net/notes-2007.html#20-09-2007
> http://landley.net/notes-2007.html#20-11-2007
> http://landley.net/notes-2008.html#18-03-2008
> http://landley.net/notes-2008.html#22-08-2008
> http://landley.net/notes-2008.html#23-08-2008
> http://landley.net/notes-2008.html#26-08-2008
> http://landley.net/notes-2008.html#19-09-2008
> http://landley.net/notes-2008.html#15-10-2008
> http://landley.net/notes-2008.html#11-12-2008
> http://landley.net/notes-2008.html#14-12-2008
> http://landley.net/notes-2009.html#01-03-2009
> http://landley.net/notes-2009.html#02-03-2009
> http://landley.net/notes-2009.html#17-08-2009
> http://landley.net/notes-2009.html#15-12-2009
> http://landley.net/notes-2010.html#10-01-2010
> http://landley.net/notes-2010.html#19-07-2010
> http://landley.net/notes-2010.html#09-08-2010
> http://landley.net/notes-2011.html#20-05-2011
> http://landley.net/notes-2011.html#15-08-2011
> http://landley.net/notes-2011.html#13-11-2011
> http://landley.net/notes-2011.html#16-12-2011
>
> I'm assembling a list of topics at
> http://landley.net/toybox/oldlicense.txt that I should cover in a once
> and for all writeup.
>
> (Oh, and
>
> http://archive.org/details/EbenMoglenLectureEdinburghJune2007StreamingVideo384kbits
> is really good, even if he's on the other side.)
>
> > I'll be interested to read it! :)
>
> Above is the slush pile. (And not even the _entire_ slush pile.) I need
> to digest that into something manageable.
>
> > Dave
> >
> >
> >
> > _______________________________________________
> > Aboriginal mailing list
> > Aboriginal at lists.landley.net
> > http://lists.landley.net/listinfo.cgi/aboriginal-landley.net
>
>
> --
> GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
> Either it's "mere aggregation", or a license violation.  Pick one.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20120726/48e37b85/attachment.htm>

From rob at landley.net  Thu Jul 26 13:28:31 2012
From: rob at landley.net (Rob Landley)
Date: Thu, 26 Jul 2012 08:28:31 -0500
Subject: [Aboriginal] powerpc status
Message-ID: <501145FF.2070101@landley.net>

The powerpc target builds to completion on my ubuntu netbook. The same
code (same build directory rsynced over there) doesn't on the Fedora server.

I really hate Fedora. I'm temted to cut a release _without_ fixing the
Fedora problems, and make that a goal for next release, except my goals
for next release are:

1) Add toybox as an alternative to busybox.
2) Add musl as an alternative to uClibc.

In the long run, I'm probably migrating to both. Toybox for obvious
reasons, and musl because uClibc's spent 6 years more or less stagnant,
doesn't think being able to bisect a bug between two release versions is
something anyone would ever want to do, and is too complicated for me to
really fix the weirdness in it myself. (I read through chunks of the
pthreads code: it looks like they copied buckets gnu crap verbatim.)

Musl is smaller and simpler and has a responsive maintainer, and is
adding new architectures as we speak, so asking for things like s390
support aren't as big a deal there. (QEMU supports it, my old GPLv2
gcc/binutils toolchain supports it, so why not?)

Plus the bsd vs gpl thing: musl could reasonably be deployed on Android
as a replacement for bionic. uClibc can't: no GPL in userspace.

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.

 1343309311.0

From rob at landley.net  Fri Jul 27 13:45:46 2012
From: rob at landley.net (Rob Landley)
Date: Fri, 27 Jul 2012 08:45:46 -0500
Subject: [Aboriginal] [PATCH] Fixing sh4 serial abort
Message-ID: <50129B8A.3060102@landley.net>

If you grab the current aboriginal linux build scripts:

  http://landley.net/hg/aboriginal/archive/tip.tar.bz2

And "./build.sh sh4", then cd to build/system-image-sh4 and
"./run-emulator.sh" you get this:

  sh_serial: unsupported read from 0x18
  Aborted

The bug was triggered by linux kernel commit 73c3d53f38e0a8e6 back
between v3.2 and v3.3, which did this:

--- a/drivers/tty/serial/sh-sci.c
+++ b/drivers/tty/serial/sh-sci.c
@@ -1771,18 +1771,25 @@ static void sci_set_termios(struct uart_port
*port, struct ktermios *termios,

 	sci_init_pins(port, termios->c_cflag);

-	if (s->cfg->capabilities & SCIx_HAVE_RTSCTS) {
-		reg = sci_getreg(port, SCFCR);
-		if (reg->size) {
-			unsigned short ctrl;
+	reg = sci_getreg(port, SCFCR);
+	if (reg->size) {
+		unsigned short ctrl = sci_in(port, SCFCR);

-			ctrl = sci_in(port, SCFCR);
+		if (s->cfg->capabilities & SCIx_HAVE_RTSCTS) {

I.E. sci_getreg(port, SCFCR) move to before checking whether or not
we'll ever possibly use the result. SCFCR is 0x18 and QEMU calls abort()
on an attempt to read from an unimplemented register.

I can patch the kernel to work around this (and probably will for this
release), but the _proper_ fix is to get qemu not to abort on a register
read that works fine if it just returns 0.

It turns out the qemu fix (in current git) is just:

--- a/hw/sh_serial.c
+++ b/hw/sh_serial.c
@@ -248,11 +248,9 @@ static uint64_t sh_serial_read(void *opaque,
target_phys
                     s->flags &= ~SH_SERIAL_FLAG_RDF;
             }
             break;
-#if 0
         case 0x18:
             ret = s->fcr;
             break;
-#endif
         case 0x1c:
             ret = s->rx_cnt;
             break;

(It can also just do ret = 0; and that works too. Or comment out the
abort() near the end of the function.)

Doing that, qemu boots the sh4 system image to a shell prompt, and the
result compiles "hello world" natively.

By the way, this board emulation (r2d) only has 64 megs ram. Is there an
easy way to get 256 megs out of it?

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.

 1343396746.0

From peter.maydell at linaro.org  Fri Jul 27 14:32:01 2012
From: peter.maydell at linaro.org (Peter Maydell)
Date: Fri, 27 Jul 2012 15:32:01 +0100
Subject: [Aboriginal] [Qemu-devel] [PATCH] Fixing sh4 serial abort
In-Reply-To: <50129B8A.3060102@landley.net>
References: <50129B8A.3060102@landley.net>
Message-ID: <CAFEAcA_qgzOjTRDeaaWxT+_hYu_iqRR6fY9ssMOm_qD6W7ZJXA@mail.gmail.com>

On 27 July 2012 14:45, Rob Landley <rob at landley.net> wrote:
> I.E. sci_getreg(port, SCFCR) move to before checking whether or not
> we'll ever possibly use the result. SCFCR is 0x18 and QEMU calls abort()
> on an attempt to read from an unimplemented register.
>
> I can patch the kernel to work around this (and probably will for this
> release), but the _proper_ fix is to get qemu not to abort on a register
> read that works fine if it just returns 0.

The thing this analysis is missing is any examination of the question
"what is the hardware we are modelling documented to do?". If the hardware
is documented to have a readable register here, QEMU should be fixed.
If it is not then the kernel is buggy and should be fixed.

As it happens, the "SH7214 Group, SH7216 Group User?s Manual: Hardware"
(which I think is the right doc for this) says the register is r/w,
so I think your suggested patch is correct.

(Aborting is a little unfriendly but our logging infrastructure
for "guest did something wrong" is not great, unfortunately.)

There are an awful lot of "#if 0"s in that source file...

-- PMM

 1343399521.0

From peter.maydell at linaro.org  Fri Jul 27 17:28:15 2012
From: peter.maydell at linaro.org (Peter Maydell)
Date: Fri, 27 Jul 2012 18:28:15 +0100
Subject: [Aboriginal] [Qemu-devel] [PATCH] Fixing sh4 serial abort
In-Reply-To: <5012CCD5.7020205@landley.net>
References: <50129B8A.3060102@landley.net>
	<CAFEAcA_qgzOjTRDeaaWxT+_hYu_iqRR6fY9ssMOm_qD6W7ZJXA@mail.gmail.com>
	<5012CCD5.7020205@landley.net>
Message-ID: <CAFEAcA8=UgFQQikkwfv_da=66-5YtHOSx9w2woTRPSpQGrbHhQ@mail.gmail.com>

On 27 July 2012 18:16, Rob Landley <rob at landley.net> wrote:
> On 07/27/2012 09:32 AM, Peter Maydell wrote:
>> On 27 July 2012 14:45, Rob Landley <rob at landley.net> wrote:
>>> I.E. sci_getreg(port, SCFCR) move to before checking whether or not
>>> we'll ever possibly use the result. SCFCR is 0x18 and QEMU calls abort()
>>> on an attempt to read from an unimplemented register.
>>>
>>> I can patch the kernel to work around this (and probably will for this
>>> release), but the _proper_ fix is to get qemu not to abort on a register
>>> read that works fine if it just returns 0.
>>
>> The thing this analysis is missing is any examination of the question
>> "what is the hardware we are modelling documented to do?".
>
> Given that 3.3, 3.4, and 3.5 kernels have already shipped with this, I'm
> guessing "not immediately crash"?

I said "documented", not "what it happens to do in practice".

> Then again I can't really criticize sh4 for multiple kernel releases not
> working under qemu and nobody noticing when _arm_ currenty has a similar
> problem. The arm versatile board's scsi emulation was broken by linux
> commit 4d5fc58dbe34 in 3.4 (yanking the versatile's mach/io.h when the
> default one breaks PCI), and then before _that_ got reverted (in commit
> 9b0f7e399238c6) this commit went in:
>
> commit 1bc39ac5dab265b76ce6e20d6c85f900539fd190
> Author: Russell King <rmk+kernel at arm.linux.org.uk>
> Date:   Sat Mar 10 11:32:34 2012 +0000
>
>     ARM: PCI: versatile: fix PCI interrupt setup
>
>     This is at odds with the documentation in the file; it says pin 1 on
>     slots 24,25,26,27 map to IRQs 27,28,29,30, but the function will
>     always be entered with slot=0 due to the lack of swizzle function.
>     Fix this function to behave as the comments say, and use the
>     standard PCI swizzle.
>
>     Signed-off-by: Russell King <rmk+kernel at arm.linux.org.uk>
>
> So the arm maintainer noticed a place where the code didn't match the
> documentation, changed the code to match the docs, and result doesn't
> work under qemu's versatile board emulation. But at least 3.5 doesn't
> work for a _different_ reason than 3.4 didn't work, so there's that.

PCI on versatilepb is hopelessly broken -- the only thing the kernel
code does (or did) is work on QEMU. Arnd Bergmann had some kernel
patches which fixed all that so it worked on hardware, but they never
got applied. It looks like Russell has now belatedly noticed a
subset of that wrongness.

> I hadn't reported this one yet because I still haven't root caused it,
> just bisected to find the break. I know reverting the IRQ assignment
> line in 3.5 doesn't fix it, which implies the "swizzle" bit is to blame
> (which seems ot have something to do with PCI bridges), and thus calling
> the default function instead of calling no function breaks qemu's
> versatile board emulation.

1. Find Arnd's kernel patches and see what of them still need to
be applied (http://comments.gmane.org/gmane.linux.ports.arm.kernel/93393)
2. Get kernel working on real hardware
3. Submit qemu patches to fix our versatilepb PCI emulation to
match hardware

If you like you can do 3 first and I'll happily apply those patches
regardless of whether anybody cares to fix the kernel.

I'm afraid I don't have a great deal of time for versatilepb
emulation fixes because it's an incredibly ancient devboard.
vexpress breakage will get more attention from me (though I
don't habitually test new kernels on qemu so I won't necessarily
notice bugs unless my attention is drawn to them.)

-- PMM

 1343410095.0

From rob at landley.net  Fri Jul 27 17:16:05 2012
From: rob at landley.net (Rob Landley)
Date: Fri, 27 Jul 2012 12:16:05 -0500
Subject: [Aboriginal] [Qemu-devel] [PATCH] Fixing sh4 serial abort
In-Reply-To: <CAFEAcA_qgzOjTRDeaaWxT+_hYu_iqRR6fY9ssMOm_qD6W7ZJXA@mail.gmail.com>
References: <50129B8A.3060102@landley.net>
	<CAFEAcA_qgzOjTRDeaaWxT+_hYu_iqRR6fY9ssMOm_qD6W7ZJXA@mail.gmail.com>
Message-ID: <5012CCD5.7020205@landley.net>

On 07/27/2012 09:32 AM, Peter Maydell wrote:
> On 27 July 2012 14:45, Rob Landley <rob at landley.net> wrote:
>> I.E. sci_getreg(port, SCFCR) move to before checking whether or not
>> we'll ever possibly use the result. SCFCR is 0x18 and QEMU calls abort()
>> on an attempt to read from an unimplemented register.
>>
>> I can patch the kernel to work around this (and probably will for this
>> release), but the _proper_ fix is to get qemu not to abort on a register
>> read that works fine if it just returns 0.
> 
> The thing this analysis is missing is any examination of the question
> "what is the hardware we are modelling documented to do?".

Given that 3.3, 3.4, and 3.5 kernels have already shipped with this, I'm
guessing "not immediately crash"?

Then again I can't really criticize sh4 for multiple kernel releases not
working under qemu and nobody noticing when _arm_ currenty has a similar
problem. The arm versatile board's scsi emulation was broken by linux
commit 4d5fc58dbe34 in 3.4 (yanking the versatile's mach/io.h when the
default one breaks PCI), and then before _that_ got reverted (in commit
9b0f7e399238c6) this commit went in:

commit 1bc39ac5dab265b76ce6e20d6c85f900539fd190
Author: Russell King <rmk+kernel at arm.linux.org.uk>
Date:   Sat Mar 10 11:32:34 2012 +0000

    ARM: PCI: versatile: fix PCI interrupt setup

    This is at odds with the documentation in the file; it says pin 1 on
    slots 24,25,26,27 map to IRQs 27,28,29,30, but the function will
    always be entered with slot=0 due to the lack of swizzle function.
    Fix this function to behave as the comments say, and use the
    standard PCI swizzle.

    Signed-off-by: Russell King <rmk+kernel at arm.linux.org.uk>

So the arm maintainer noticed a place where the code didn't match the
documentation, changed the code to match the docs, and result doesn't
work under qemu's versatile board emulation. But at least 3.5 doesn't
work for a _different_ reason than 3.4 didn't work, so there's that.

I dunno if this is a qemu bug or a kernel bug, but I _do_ know that I'm
far more interested in getting it to work with existing software than
getting it to match the docs.

I hadn't reported this one yet because I still haven't root caused it,
just bisected to find the break. I know reverting the IRQ assignment
line in 3.5 doesn't fix it, which implies the "swizzle" bit is to blame
(which seems ot have something to do with PCI bridges), and thus calling
the default function instead of calling no function breaks qemu's
versatile board emulation.

But that part isn't trivially reverted in 3.5 becauase they "cleaned up"
the code so you now get the default behavior without asking and I
haven't dug deep enough to figure out what's actually going on yet.

> If the hardware
> is documented to have a readable register here, QEMU should be fixed.
> If it is not then the kernel is buggy and should be fixed.

The sh4 Linux architecture is currently maintained by Renesas
developers, and they're just as good a maintainer of the dreamcast-era
stuff as Oracle is of Sun workstations and HP/Compaq was of the Microvax.

They have explicitly told me they aren't interested in anyone who isn't
a Renesas customer, and that since attempting to run sh4 under qemu or
on a dreamcast isn't what they're paid to support, they therefore do not
care about it.

Pointing out to them that they broke older versions of stuff has never
worked for me. You're welcome to try to engage them, but personally I
treat the sh4 changes as read only at this point and carry local patches
to revert the really stupid bits.

> As it happens, the "SH7214 Group, SH7216 Group User?s Manual: Hardware"
> (which I think is the right doc for this) says the register is r/w,
> so I think your suggested patch is correct.

Yay!

> (Aborting is a little unfriendly but our logging infrastructure
> for "guest did something wrong" is not great, unfortunately.)

My first fix was just replacing the abort with assigning the value to 0,
but that left debug output spewing a bit.

> There are an awful lot of "#if 0"s in that source file...

This one was put in by Thiemo Suefer. He didn't get to finish cleaning
up before he died.

I'm aware there's not a huge amount of interest in Linux on sh4, but my
aboriginal linux project is aimed at getting the same basic native
development environment working on as many different architectures as
possible, for things like automated cross-platform regression testing.
(I was thinking "userspace packages", but if I can set up a cron job
doing nightly builds of git maybe some of this crud would be caught
earlier. Pity I have NO free time. Writing this at work over lunch.)

So if qemu emulates it, I'm interested in getting a linux running on it
with a native compiler. Which means I'm perpetually out of my depth, but
oh well...

> -- PMM

Thanks for the review,

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.

 1343409365.0