From rob at landley.net  Fri Apr  1 13:11:23 2011
From: rob at landley.net (Rob Landley)
Date: Fri, 01 Apr 2011 08:11:23 -0500
Subject: [Aboriginal] Aboriginal Linux
Message-ID: <4D95CEFB.5080807@landley.net>

On 03/31/2011 02:19 PM, Alessio Igor Bogani wrote:
> Dear Mr. Landley,
> 
> I would want let you know that system-image-arm6l doesn't boot in my qemu:
> 
> abogani at picasso:~/WIP/system-image-armv6l$ qemu-system-arm -version
> QEMU emulator version 0.14.50 (Debian 0.14.50-2011.03-1-0ubuntu1),
> Copyright (c) 2003-2008 Fabrice Bellard

Yeah, it happened when I upgraded the kernel.  Fixing this has been on
my todo list.

Lemme see...  according to "hg log -v" 1322 introduced 2.6.37, 1265 was
2.6.36, and 1231 was 2.6.35...  QEMU could boot 2.6.35 fine but 2.6.36
doesn't boot.  (Presumably it does so on real hardware...)

It's related to my patching of the Arm versatile board to let you select
which processor to stick in it.  QEMU allows you to stick an armv6 cpu
in a versatile board via the "-cpu arm1136" option, but this doesn't get
a lot of testing and a kernel upgrade broke it.  I need to bisect to
find which commit introduced the breakage and see what I can do about it.

(Alternately, I could try one of the other board emulations QEMU does now.)

> abogani at picasso:~/WIP/system-image-armv6l$ ./run-emulator.sh
> qemu-system-arm: pci_add_option_rom: failed to find romfile "pxe-rtl8139.bin"
> Uncompressing Linux... done, booting the kernel.
> qemu: fatal: Unimplemented cp15 register write (c13, c0, {0, 3})
> 
> R00=c02896c0 R01=c0286000 R02=c7828000 R03=00000000
> R04=00008197 R05=ffff0fff R06=00000017 R07=c78242e0
> R08=c028caa8 R09=000003f0 R10=c0286000 R11=c0287fb4
> R12=c0286044 R13=c0287f88 R14=c0217920 R15=c001dc94
> PSR=00000153 ---- A svc32
> ./run-emulator.sh: line 3: 22118 Aborted                 (core dumped)
> qemu-system-arm -M versatilepb -cpu arm1136 -nographic -no-reboot
> -kernel "zImage-armv6l" $WITH_HDC $WITH_HDB -hda "image-armv6l.sqf"
> -append "root=/dev/sda rw init=/sbin/init.sh panic=1
> PATH=$DISTCC_PATH_PREFIX/bin:/sbin console=ttyAMA0 HOST=armv6l
> $KERNEL_EXTRA" $QEMU_EXTRA -net nic,model=rtl8139 -net user
> 
> Instead system-image-arm4l works as expected.

armv4l is OABI, you might want armv4tl or armv5l.  (Those also currently
work, it's just armv6l that qemu barfs on.)

> Thank you very much!
> 
> Ciao,
> Alessio
> 

Rob

 1301663483.0

From onefang at gmail.com  Fri Apr  1 14:04:43 2011
From: onefang at gmail.com (David Seikel)
Date: Sat, 2 Apr 2011 00:04:43 +1000
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <4D94771C.7070708@landley.net>
References: <20110331180847.1e70e680@U64-desktop>
	<4D94771C.7070708@landley.net>
Message-ID: <20110402000443.2caa5c65@U64-desktop>

On Thu, 31 Mar 2011 07:44:12 -0500 Rob Landley <rob at landley.net> wrote:

> On 03/31/2011 03:08 AM, David Seikel wrote:
> > ulfs is my effort to shrink LFS down, mostly to suit my own
> > purposes, by using the stuff that already exists in Aboriginal
> > linux.
> 
> I need better documentation, I'm just not sure where to put it.

I saw your tags thingy, but it did not match up with what I ended up
with.  Or saw how your tags should be used.
 
> However, that's the sort of thing that could easily open "the distro
> can of worms" which I'm trying very hard not to do with this
> project.  Of course if somebody ELSE wanted to take that over... :)

I'm not volunteering, I got enough work to do.  Only dabbling in the
stuff I need to, and contributing back what I can.

> > no need to replace the tool chain we already got,
> 
> In the LFS bootstrap script I never bothered to build the new
> toolchain. I built its prerequisites, but skipped gcc and binutils.
> (I _do_ rebuilt make, for no readily apparent reason.)

Yes, I built make to, then removed it after the first draft of my email,
which is why it was still on my list in the email.  I forgot to remove
it from the list.

> Bash I'm installing a newer version of.  The old 2.04b is easy to
> build and less than 1/3 of the size of the current monstrosity, and I
> really want to replace it all with busybox hush.  But you need
> current bash in order to run gentoo's portage, it uses stuff like ~=
> that old bash hasn't got, and depends on the subtly changed quoting
> rules...

Gentoo I'm not interested in.  LFS and BLFS I am interested in, but
only for those packages I need for the current project.  A later 
project that will grow out of this will need more from BLFS.  I still
wont be interested in gentoo.

> Readline there's a bsd-licensed replacement I'd prefer to use out of
> sheer spite for the FSF, and idealy busybox should export that
> functionality anyway.

I'd like to look at that readline replacement, though I'd be just as
happy to not need it for my current project.

> Don't get me started on autoconf/automake.

Preaching to the converted, I was glad to get rid of it.

> pkg-config is its own horror (you do NOT improve on the rpm vs dpkg vs
> ebuild situation by ADDING YET ANOTHER... grumble) and either not
> having it installed or setting PKG_CONFIG=/bin/true is my preferred
> choice, butI got tired of fighting with some package builds.

The UI library I want to use makes heavy use of pkg-config, so for me
it stays.

> But the point is, lots of BLFS work just fine under uClibc.  And I'm
> hoping to get more under uClibc-1.0.  (In fact, once we've got NTPL
> and TLS I'd like to treat aborigininal Linux as the /tools system and
> build a new glibc based system natively, just so I don't ever have to
> worry about cross compiling glibc.  It requires perl to build.  You
> may barf now.)

I have a strong stomach.  :-P

Though uClibc is good for me.

> You have to remove make from the package-list (it's currently in
> there), but yes, you shouldn't need it.

Left in the email by mistake.

> > First problem, iana-etc needs the real gawk, it uses fancy options
> > that the busybox awk does not support.  So put gawk back in.
> 
> Poke the busybox gawk maintainer about that.

I may just cut iana-etc down to the few ports I actually need for this
project.  The next project will need it for sure.  I've not actually
looked at what iana-etc uses gawk for, but it does smell of overkill to
just create two text files.  Why not just ship the two text files?

> > Second problem, autoconf stops part way through asking the user a
> > question when using busybox instead of coreutils.  A quick
> > investigation seems to point to the busybox mv command needing to
> > know if it's OK to overwrite a file, when the coreutils one just
> > overwrites it without asking.  There may in fact be some deeper
> > root cause for this problem, I did not look deeper.  See next step
> > for why.
> 
> I'd poke busybox about that.

Well, it became moot the moment I removed autoconf from the mix.  Also
no point poking busybox unless I bother to dig deeper, which I have no
intention of doing for a package I'm happy I don't have to include
anyway.

> Not that I really have time to do so, but eh.  If I copied hush and
> started cleaning that up and extending it, I might get a reasonable
> bash replacement THAT way...

That would be good.

> > Grub was left off for the same reason it was left out of
> > lfs-bootstrap.
> 
> I've never really addressed bootloaders, or installing stuff on real
> hardware.  It's too system-specific, and QEMU's -kernel option allows
> me to handwave past the issue entirely for my use cases. :)

Though in the end I will use grub legacy as it's now known in this
project.  The real hardware will have to be booted somehow.

> > Bash and readline where removed.  We already have an older bash as
> > part of aboriginal linux.
> 
> Which doesn't work with portage, so I needed newer bash as part of the
> gentoo bootstrap anyway.

Already mentioned I don't need to care for gentoo.  Not like anyone is
using LFS to bootstrap gentoo, and this is the LFS bootstrap we are
talking about.

> > Readline is only in LFS as a dependency of bash.
> > Time will tell if this is a bad move.
> 
> I was erring on the side of building more packages rather than less on
> the theory that if I could get it to build, anything that actually did
> need it could then use it, and driving more cybermen across the board
> would find more digits of pi, or something like that.  ("Used to drive
> sheep across minefields.  The principle's the same.")

True.  On the other hand, one of the goals of this project of mine is
to provide a government testing lab with something for them to audit,
including the development system that we use to build it all, so that
the lab can build it to.  The less they have to deal with, the better
we will be.  So at a later stage I'll very carefully pull out as much
stuff as I can to pare it down to the bare minimum.  Simple
minimisations are good enough for this early in the project.  Get it to
work first, then optimise away more crap later.

It's a personal goal of mine that the development system we hand to the
audit lab is just the device itself, and a hard drive they plug into
the device that has the development environment on it.  This is why
Aboriginal Linux's ability to compile itself is important to me.

> yacc/bison is something mathematicians like and which I personally
> find conceptually unpleasant.

There might be something using bison that is important to me later.
I'll know for sure later.

> > I guess iconv was left out as it's just bloat that is not needed if
> > you are not doing any internationalization, for the same reason
> > gettext-stub is used instead of gettext in lfs-bootstrap.  I think
> > gentoo have a stub version of iconv they are working on, but the
> > above was easy to do and worked for me.  I'm not doing
> > internationalization either.
> 
> My 2010 blog (http://landley.net/notes-2010.html) has a lot of hits
> for iconv.

I may grep that soon.

> On the Hexagon I built the x11 client libraries and managed to build
> xeyes and xboard and rxvt and a couple other things.  There was some
> hacking around uClibc but they were using 0.9.30 rather than 0.9.31 so
> lots of it was backporting stuff...

Trying to avoid X this time around, that's for the next project.  For
this project I think the kernels framebuffer is good enough, and helps
a lot with the "minimal code for the audit lab to worry about" goal
nicely.  No need to worry about adding 50 X packages when all I need is
a framebuffer that already comes with the kernel.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110402/b38321ac/attachment.sig>

From rlandley at parallels.com  Sat Apr  2 06:15:17 2011
From: rlandley at parallels.com (Rob Landley)
Date: Sat, 2 Apr 2011 01:15:17 -0500
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <20110402000443.2caa5c65@U64-desktop>
References: <20110331180847.1e70e680@U64-desktop>	<4D94771C.7070708@landley.net>
	<20110402000443.2caa5c65@U64-desktop>
Message-ID: <4D96BEF5.5040401@parallels.com>

On 04/01/2011 09:04 AM, David Seikel wrote:
> On Thu, 31 Mar 2011 07:44:12 -0500 Rob Landley <rob at landley.net> wrote:
> 
>> On 03/31/2011 03:08 AM, David Seikel wrote:
>>> ulfs is my effort to shrink LFS down, mostly to suit my own
>>> purposes, by using the stuff that already exists in Aboriginal
>>> linux.
>>
>> I need better documentation, I'm just not sure where to put it.
> 
> I saw your tags thingy, but it did not match up with what I ended up
> with.  Or saw how your tags should be used.
>  
>> However, that's the sort of thing that could easily open "the distro
>> can of worms" which I'm trying very hard not to do with this
>> project.  Of course if somebody ELSE wanted to take that over... :)
> 
> I'm not volunteering, I got enough work to do.  Only dabbling in the
> stuff I need to, and contributing back what I can.

This is why I want to do distro bootstrapping.  Maintaining package
repositories is something _other_ people do, and do it well, and I want
to leverage that work, not duplicate it.

Even LFS (and BLFS) are pretty much "repositories somebody else
maintains".  They're as simple as you can get doing that (fixed package
selection), but I don't have to work out what depends on what and how to
configure invoke the builds.  (I just have to debug it. :)

>>> no need to replace the tool chain we already got,
>>
>> In the LFS bootstrap script I never bothered to build the new
>> toolchain. I built its prerequisites, but skipped gcc and binutils.
>> (I _do_ rebuilt make, for no readily apparent reason.)
> 
> Yes, I built make to, then removed it after the first draft of my email,
> which is why it was still on my list in the email.  I forgot to remove
> it from the list.
> 
>> Bash I'm installing a newer version of.  The old 2.04b is easy to
>> build and less than 1/3 of the size of the current monstrosity, and I
>> really want to replace it all with busybox hush.  But you need
>> current bash in order to run gentoo's portage, it uses stuff like ~=
>> that old bash hasn't got, and depends on the subtly changed quoting
>> rules...
> 
> Gentoo I'm not interested in.  LFS and BLFS I am interested in, but
> only for those packages I need for the current project.  A later 
> project that will grow out of this will need more from BLFS.  I still
> wont be interested in gentoo.

Debian/Ubuntu and Fedora/Pointy Hair Linux are the two biggest
repositories.  Gentoo is probably third, although Arch Linux is creeping
up on it.

In theory Gentoo's emphasis on building from source made it a good
candidate for bootstrapping systems on arbitrary targets.  In practice
the way they set up their repository individual annotates every single
package with every single platform it's been tested to run on.  (To add
a new platform, you have to touch every package ebuild file in the whole
tree.)

That's insanely stupid for native builds on arbitrary targets you've
already bootstrapped on.  In order to go forward, I need to REDESIGN THE
PORTAGE TREE and push the changes upstream.

>> Readline there's a bsd-licensed replacement I'd prefer to use out of
>> sheer spite for the FSF, and idealy busybox should export that
>> functionality anyway.
> 
> I'd like to look at that readline replacement, though I'd be just as
> happy to not need it for my current project.

There's a couple of them, but this one looks reasonable at first glance:

  https://github.com/antirez/linenoise

>> Don't get me started on autoconf/automake.
> 
> Preaching to the converted, I was glad to get rid of it.
> 
>> pkg-config is its own horror (you do NOT improve on the rpm vs dpkg vs
>> ebuild situation by ADDING YET ANOTHER... grumble) and either not
>> having it installed or setting PKG_CONFIG=/bin/true is my preferred
>> choice, butI got tired of fighting with some package builds.
> 
> The UI library I want to use makes heavy use of pkg-config, so for me
> it stays.

You can't solve having too many layers of indirection by adding another
layer of indirection.

>>> First problem, iana-etc needs the real gawk, it uses fancy options
>>> that the busybox awk does not support.  So put gawk back in.
>>
>> Poke the busybox gawk maintainer about that.
> 
> I may just cut iana-etc down to the few ports I actually need for this
> project.  The next project will need it for sure.  I've not actually
> looked at what iana-etc uses gawk for, but it does smell of overkill to
> just create two text files.  Why not just ship the two text files?

I know.  (Haven't looked at it since october, I think there was some
kind of hammer I could hit it with to make it stop...)

>>> Second problem, autoconf stops part way through asking the user a
>>> question when using busybox instead of coreutils.  A quick
>>> investigation seems to point to the busybox mv command needing to
>>> know if it's OK to overwrite a file, when the coreutils one just
>>> overwrites it without asking.  There may in fact be some deeper
>>> root cause for this problem, I did not look deeper.  See next step
>>> for why.
>>
>> I'd poke busybox about that.
> 
> Well, it became moot the moment I removed autoconf from the mix.  Also
> no point poking busybox unless I bother to dig deeper, which I have no
> intention of doing for a package I'm happy I don't have to include
> anyway.

See yesterday's grumble about restarting toybox.  Not much point in
doing it unless I can _replace_ busybox, but if I change my dev strategy
to porting cleaned-up versions of busybox applets to toybox commands...

Don't have time at the moment, though.

>> Not that I really have time to do so, but eh.  If I copied hush and
>> started cleaning that up and extending it, I might get a reasonable
>> bash replacement THAT way...
> 
> That would be good.

That by _itself_ is probably as big a project as the rest of toybox.

>>> Grub was left off for the same reason it was left out of
>>> lfs-bootstrap.
>>
>> I've never really addressed bootloaders, or installing stuff on real
>> hardware.  It's too system-specific, and QEMU's -kernel option allows
>> me to handwave past the issue entirely for my use cases. :)
> 
> Though in the end I will use grub legacy as it's now known in this
> project.  The real hardware will have to be booted somehow.

Yup, but maintaining a hardware list (how to install and boot on the
zillions of boards out there) is beyond the scope of the Aboriginal
Linux project the same way maintaining a package repositor is.

It would be great if somebody did this, but it's an orthogonal layer.

>>> Bash and readline where removed.  We already have an older bash as
>>> part of aboriginal linux.
>>
>> Which doesn't work with portage, so I needed newer bash as part of the
>> gentoo bootstrap anyway.
> 
> Already mentioned I don't need to care for gentoo.  Not like anyone is
> using LFS to bootstrap gentoo, and this is the LFS bootstrap we are
> talking about.

Actually I was pondering using LFS to bootstrap gentoo (since I'd
already done the work, might as well leverage it), but yeah.  Not in
your prerequisite list.

I designed the LFS bootstrap to be as modular as I could.  You can run
each package build individually:

  /mnt/build-one-package.sh packagename

Fun for debugging.  You can also:

  source /mnt/functions.sh; do_in_chroot /home/chrootdir /bin/ash

Which is clunky and I need to make a separate /usr/src/do-in-chroot.sh
as part of the base script.  (One that autodetects whether the target
directory exists, and just redoes the bind mounts if so.)

(Note my general rule of thumb for underscores vs dashes: do_in_chroot
is a shell function, do-in-chroot.sh is a shell script.  Another thing I
try to be consistent about, but haven't really documented anywhere.  I'm
not the only one doing that, though...)

>>> Readline is only in LFS as a dependency of bash.
>>> Time will tell if this is a bad move.
>>
>> I was erring on the side of building more packages rather than less on
>> the theory that if I could get it to build, anything that actually did
>> need it could then use it, and driving more cybermen across the board
>> would find more digits of pi, or something like that.  ("Used to drive
>> sheep across minefields.  The principle's the same.")
> 
> True.  On the other hand, one of the goals of this project of mine is
> to provide a government testing lab with something for them to audit,
> including the development system that we use to build it all, so that
> the lab can build it to.  The less they have to deal with, the better
> we will be.  So at a later stage I'll very carefully pull out as much
> stuff as I can to pare it down to the bare minimum.  Simple
> minimisations are good enough for this early in the project.  Get it to
> work first, then optimise away more crap later.

I hope the orthogonal layers of the build system are reasonable to audit
separately. :)

> It's a personal goal of mine that the development system we hand to the
> audit lab is just the device itself, and a hard drive they plug into
> the device that has the development environment on it.  This is why
> Aboriginal Linux's ability to compile itself is important to me.

Important to me too.  You haven't got a real build environment if it
doesn't self-bootstrap.

>> yacc/bison is something mathematicians like and which I personally
>> find conceptually unpleasant.
> 
> There might be something using bison that is important to me later.
> I'll know for sure later.

You can always add more packages later.

>> On the Hexagon I built the x11 client libraries and managed to build
>> xeyes and xboard and rxvt and a couple other things.  There was some
>> hacking around uClibc but they were using 0.9.30 rather than 0.9.31 so
>> lots of it was backporting stuff...
> 
> Trying to avoid X this time around, that's for the next project.  For
> this project I think the kernels framebuffer is good enough, and helps
> a lot with the "minimal code for the audit lab to worry about" goal
> nicely.  No need to worry about adding 50 X packages when all I need is
> a framebuffer that already comes with the kernel.

It was more like 35.

(And they all pretty much built the same way, too.  For some obscure
reason the x.org guys decided to break the one big xfree86 packages into
three dozen way to small packages.  I'm guessing they got pestered about
modularity once too often, and were trying to prove a point by making it
virtually unusable in the other direction.)

Rob

 1301724917.0

From onefang at gmail.com  Tue Apr  5 10:24:05 2011
From: onefang at gmail.com (David Seikel)
Date: Tue, 5 Apr 2011 20:24:05 +1000
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <4D96BEF5.5040401@parallels.com>
References: <20110331180847.1e70e680@U64-desktop>
	<4D94771C.7070708@landley.net>
	<20110402000443.2caa5c65@U64-desktop>
	<4D96BEF5.5040401@parallels.com>
Message-ID: <20110405202405.4d285901@U64-desktop>

The rest of that I'll reply to later, just want to get something off my
chest before sleeping.

On Sat, 2 Apr 2011 01:15:17 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> >> Readline there's a bsd-licensed replacement I'd prefer to use out
> >> of sheer spite for the FSF, and idealy busybox should export that
> >> functionality anyway.  
> > 
> > I'd like to look at that readline replacement, though I'd be just as
> > happy to not need it for my current project.  
> 
> There's a couple of them, but this one looks reasonable at first
> glance:
> 
>   https://github.com/antirez/linenoise

Turns out that lua needs readline.  Lua also needs ncurses.  Lua itself
is needed by the UI library I'm using (EFL at
http://enlightenment.org/ ).  I'm guessing that lua needs readline and
ncurses for it's stand alone lua shell.  Not something I need for the
final product, but likely it will come in handy for debugging.  I'll
see if I can patch that shell out when time comes for me to shrink the
result down to the bare minimum.  For now it stays.  I'm looking
forward to learning lua and using it in this project.

I remember you where thinking about using lua for toybox.  You also
would not need a stand alone lua shell, so you might be happy to see
that patch when I make it.

BTW, is anyone else on this list, or did you loose them all and I'm
the first to find it?  Seems awfully quiet in here.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110405/8111767e/attachment.sig>

From rlandley at parallels.com  Tue Apr  5 11:49:48 2011
From: rlandley at parallels.com (Rob Landley)
Date: Tue, 5 Apr 2011 06:49:48 -0500
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <20110405202405.4d285901@U64-desktop>
References: <20110331180847.1e70e680@U64-desktop>	<4D94771C.7070708@landley.net>	<20110402000443.2caa5c65@U64-desktop>	<4D96BEF5.5040401@parallels.com>
	<20110405202405.4d285901@U64-desktop>
Message-ID: <4D9B01DC.7050102@parallels.com>

On 04/05/2011 05:24 AM, David Seikel wrote:
> The rest of that I'll reply to later, just want to get something off my
> chest before sleeping.
> 
> On Sat, 2 Apr 2011 01:15:17 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
>>>> Readline there's a bsd-licensed replacement I'd prefer to use out
>>>> of sheer spite for the FSF, and idealy busybox should export that
>>>> functionality anyway.  
>>>
>>> I'd like to look at that readline replacement, though I'd be just as
>>> happy to not need it for my current project.  
>>
>> There's a couple of them, but this one looks reasonable at first
>> glance:
>>
>>   https://github.com/antirez/linenoise
> 
> Turns out that lua needs readline.  Lua also needs ncurses.

I thought it was readline that needed ncurses, but I haven't really
poked at lua building recently.  (It's on my todo list...)

>  Lua itself
> is needed by the UI library I'm using (EFL at
> http://enlightenment.org/ ).  I'm guessing that lua needs readline and
> ncurses for it's stand alone lua shell.  Not something I need for the
> final product, but likely it will come in handy for debugging.  I'll
> see if I can patch that shell out when time comes for me to shrink the
> result down to the bare minimum.  For now it stays.  I'm looking
> forward to learning lua and using it in this project.

Same here.  I bought the lua book and everything.  Every time I'm in
Moscow (twice so far) I try to meet up with Alexander Gladysh (senior
Lua developer) but we have yet to actually cross paths so far.

> I remember you where thinking about using lua for toybox.  You also
> would not need a stand alone lua shell, so you might be happy to see
> that patch when I make it.

You might want to ping agladysh at gmail.com and see if there's already a
way to get it to do that.

> BTW, is anyone else on this list, or did you loose them all and I'm
> the first to find it?  Seems awfully quiet in here.

The old list went away when impactlinux.com went down, and I lost the
subscription list when that happened.  I didn't get around to setting up
a new one again until a month or so back, and I haven't cut a release
since then so people who only pop up for the releases haven't
particularly noticed the new list's existence yet. :)

Several people have subscribed and are lurking, but I still get far more
emails directly than go to this list.

Speaking of which, I got permission to cc one here a couple days ago but
I don't see it... and it's not in the web archive either.  Great,
gmail's insane spam filtering strikes again.  Second attempt, modifying
the message enough that hopefully gmail won't decide it's a duplicate...

Rob

 1302004188.0

From rob at landley.net  Tue Apr  5 11:46:33 2011
From: rob at landley.net (Rob Landley)
Date: Tue, 05 Apr 2011 06:46:33 -0500
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
Message-ID: <4D9B0119.6070500@landley.net>

I tried re-replying and gmail's INSANE spam filtering decided to filter
my OUTGOING message.  (Way to go gmail.  It's not enough that when
somebody cc's me on a list message you eat the second copy so my archive
threading is all screwed up, now you're screwing up my outgoing mail as
well.  Bravo.  Just... bravo.)

-------- Original Message --------
Subject: Re: Aboriginal Linux
Date: Fri, 01 Apr 2011 04:21:30 -0500
From: Rob Landley <rob at landley.net>
To: Alessio Igor Bogani <alessioigorbogani at gmail.com>

Do you mind if I repost this reply to the aboriginal linux mailing list?

On 03/31/2011 02:19 PM, Alessio Igor Bogani wrote:
> Dear Mr. Landley,
> 
> I would want let you know that system-image-arm6l doesn't boot in my qemu:
> 
> abogani at picasso:~/WIP/system-image-armv6l$ qemu-system-arm -version
> QEMU emulator version 0.14.50 (Debian 0.14.50-2011.03-1-0ubuntu1),
> Copyright (c) 2003-2008 Fabrice Bellard

Yeah, it happened when I upgraded the kernel.  Fixing this has been on
my todo list.

Lemme see...  according to "hg log -v" 1322 introduced 2.6.37, 1265 was
2.6.36, and 1231 was 2.6.35...  QEMU could boot 2.6.35 fine but 2.6.36
doesn't boot.  (Presumably it does so on real hardware...)

It's related to my patching of the Arm versatile board to let you select
which processor to stick in it.  QEMU allows you to stick an armv6 cpu
in a versatile board via the "-cpu arm1136" option, but this doesn't get
a lot of testing and a kernel upgrade broke it.  I need to bisect to
find which commit introduced the breakage and see what I can do about it.

(Alternately, I could try one of the other board emulations QEMU does now.)

> abogani at picasso:~/WIP/system-image-armv6l$ ./run-emulator.sh
> qemu-system-arm: pci_add_option_rom: failed to find romfile "pxe-rtl8139.bin"
> Uncompressing Linux... done, booting the kernel.
> qemu: fatal: Unimplemented cp15 register write (c13, c0, {0, 3})
> 
> R00=c02896c0 R01=c0286000 R02=c7828000 R03=00000000
> R04=00008197 R05=ffff0fff R06=00000017 R07=c78242e0
> R08=c028caa8 R09=000003f0 R10=c0286000 R11=c0287fb4
> R12=c0286044 R13=c0287f88 R14=c0217920 R15=c001dc94
> PSR=00000153 ---- A svc32
> ./run-emulator.sh: line 3: 22118 Aborted                 (core dumped)
> qemu-system-arm -M versatilepb -cpu arm1136 -nographic -no-reboot
> -kernel "zImage-armv6l" $WITH_HDC $WITH_HDB -hda "image-armv6l.sqf"
> -append "root=/dev/sda rw init=/sbin/init.sh panic=1
> PATH=$DISTCC_PATH_PREFIX/bin:/sbin console=ttyAMA0 HOST=armv6l
> $KERNEL_EXTRA" $QEMU_EXTRA -net nic,model=rtl8139 -net user
> 
> Instead system-image-arm4l works as expected.

armv4l is OABI, you might want armv4tl or armv5l.  (Those also currently
work, it's just armv6l that qemu barfs on.)

> Thank you very much!
> 
> Ciao,
> Alessio
> 

Rob

 1302003993.0

From alanpost at sunflowerriver.org  Tue Apr  5 15:06:43 2011
From: alanpost at sunflowerriver.org (Alan Post)
Date: Tue, 5 Apr 2011 09:06:43 -0600
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <20110405202405.4d285901@U64-desktop>
References: <20110331180847.1e70e680@U64-desktop>
	<4D94771C.7070708@landley.net>
	<20110402000443.2caa5c65@U64-desktop>
	<4D96BEF5.5040401@parallels.com>
	<20110405202405.4d285901@U64-desktop>
Message-ID: <20110405150643.GB11234@alice.local>

On Tue, Apr 05, 2011 at 08:24:05PM +1000, David Seikel wrote:
> BTW, is anyone else on this list, or did you loose them all and I'm
> the first to find it?  Seems awfully quiet in here.
> 

I'm here; I have my own Aboriginal Linux project.  It is slow
going, so I'm bursty writing the list.

-Alan
-- 
.i ma'a lo bradi ku penmi gi'e du

 1302016003.0

From onefang at gmail.com  Tue Apr  5 10:12:05 2011
From: onefang at gmail.com (David Seikel)
Date: Tue, 5 Apr 2011 20:12:05 +1000
Subject: [Aboriginal] undefined reference to `finite'
Message-ID: <20110405201205.5542b8c9@U64-desktop>

I'm very tired, and slightly ill, and I can't figure this out.

One of the packages from the UI library I want to use for this project
uses a function called finite(double).  As far as I can tell, uClibc
should supply that function, since we have DO_C99_MATH=y in its
baseconfig. It's not being found though.

It's all buried in a forest of #ifdef's that my tired mind is in no
condition to sort out.

I'm only two packages away for having all my needed libraries installed
before I can start cutting code.  Time to sleep.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110405/3307228e/attachment.sig>

From rlandley at parallels.com  Tue Apr  5 11:35:02 2011
From: rlandley at parallels.com (Rob Landley)
Date: Tue, 5 Apr 2011 06:35:02 -0500
Subject: [Aboriginal] undefined reference to `finite'
In-Reply-To: <20110405201205.5542b8c9@U64-desktop>
References: <20110405201205.5542b8c9@U64-desktop>
Message-ID: <4D9AFE66.7040906@parallels.com>

On 04/05/2011 05:12 AM, David Seikel wrote:
> I'm very tired, and slightly ill, and I can't figure this out.
> 
> One of the packages from the UI library I want to use for this project
> uses a function called finite(double).  As far as I can tell, uClibc
> should supply that function, since we have DO_C99_MATH=y in its
> baseconfig. It's not being found though.
> 
> It's all buried in a forest of #ifdef's that my tired mind is in no
> condition to sort out.
> 
> I'm only two packages away for having all my needed libraries installed
> before I can start cutting code.  Time to sleep.

Hmmm...  Posibly a question for the uClibc mailing list?

You didn't say which target you were using, so I'll poke at x86_64:

$ cd build/cross-compiler-x86_64/lib
$ grep finite *
Binary file libm-0.9.31.so matches
Binary file libm.a matches
Binary file libm.so matches
Binary file libm.so.0 matches
$ ../bin/x86_64-readelf -a libm.so | grep finite
    40: 0000000000007ad0    18 FUNC    GLOBAL DEFAULT    6 __finitef
    58: 000000000000656c    23 FUNC    GLOBAL DEFAULT    6 __finite
    63: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS s_finite.c
   102: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS s_finitef.c
   171: 0000000000007ad0    18 FUNC    LOCAL  HIDDEN    6 __GI___finitef
   177: 000000000000656c    23 FUNC    LOCAL  HIDDEN    6 __GI___finite
   290: 0000000000007ad0    18 FUNC    GLOBAL DEFAULT    6 __finitef
   308: 000000000000656c    23 FUNC    GLOBAL DEFAULT    6 __finite

It looks like the problem is that uClibc is sticking two underscores on
the front of the "finite" symbol.  I'm not sure why.  It's not doing
that with other symbols, ala:

$ ../bin/x86_64-readelf -a libm.so | grep floor
    28: 0000000000006584   299 FUNC    GLOBAL DEFAULT    6 floor
   131: 0000000000007de8    19 FUNC    GLOBAL DEFAULT    6 floorf
    64: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS s_floor.c
   162: 0000000000006584   299 FUNC    LOCAL  HIDDEN    6 __GI_floor
   278: 0000000000006584   299 FUNC    GLOBAL DEFAULT    6 floor
   381: 0000000000007de8    19 FUNC    GLOBAL DEFAULT    6 floorf

My next step would be to dig down into the uClibc source code and build
system and try to figure out where that symbol comes from and maybe if
there's some sort of other config symbol to tell it not to do that.
(The build leaves that under build/packages/uClibc by the way, extracted
with all patches applied.  The gory details are explained at
http://landley.net/aboriginal/FAQ.html#debug_source although there's a
lot of reading to do to hack through it all.  It was more elaborate to
write, and if you have suggestions for simplifying it I'm all ears.  But
understand what it _does_ first, please...)

By the way, have you read this yet?

  http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html

One of the best quick and dirty "get familiar with elf file format"
things I've seen...

Rob

 1302003302.0

From rlandley at parallels.com  Tue Apr  5 11:51:50 2011
From: rlandley at parallels.com (Rob Landley)
Date: Tue, 5 Apr 2011 06:51:50 -0500
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
Message-ID: <4D9B0256.2000502@parallels.com>

Retry sending it form my work address?  (Yes, gmail's spam filtering is
so broken I have to send this through an exchange server to get it out.
 I'm going to have to set up my own mail server again, aren't I?)

-------- Original Message --------
Subject: Re: Aboriginal Linux
Date: Fri, 01 Apr 2011 04:21:30 -0500
From: Rob Landley <rob at landley.net>
To: Alessio Igor Bogani <alessioigorbogani at gmail.com>

Do you mind if I repost this reply to the aboriginal linux mailing list?

On 03/31/2011 02:19 PM, Alessio Igor Bogani wrote:
> Dear Mr. Landley,
> 
> I would want let you know that system-image-arm6l doesn't boot in my qemu:
> 
> abogani at picasso:~/WIP/system-image-armv6l$ qemu-system-arm -version
> QEMU emulator version 0.14.50 (Debian 0.14.50-2011.03-1-0ubuntu1),
> Copyright (c) 2003-2008 Fabrice Bellard

Yeah, it happened when I upgraded the kernel.  Fixing this has been on
my todo list.

Lemme see...  according to "hg log -v" 1322 introduced 2.6.37, 1265 was
2.6.36, and 1231 was 2.6.35...  QEMU could boot 2.6.35 fine but 2.6.36
doesn't boot.  (Presumably it does so on real hardware...)

It's related to my patching of the Arm versatile board to let you select
which processor to stick in it.  QEMU allows you to stick an armv6 cpu
in a versatile board via the "-cpu arm1136" option, but this doesn't get
a lot of testing and a kernel upgrade broke it.  I need to bisect to
find which commit introduced the breakage and see what I can do about it.

(Alternately, I could try one of the other board emulations QEMU does now.)

> abogani at picasso:~/WIP/system-image-armv6l$ ./run-emulator.sh
> qemu-system-arm: pci_add_option_rom: failed to find romfile "pxe-rtl8139.bin"
> Uncompressing Linux... done, booting the kernel.
> qemu: fatal: Unimplemented cp15 register write (c13, c0, {0, 3})
> 
> R00=c02896c0 R01=c0286000 R02=c7828000 R03=00000000
> R04=00008197 R05=ffff0fff R06=00000017 R07=c78242e0
> R08=c028caa8 R09=000003f0 R10=c0286000 R11=c0287fb4
> R12=c0286044 R13=c0287f88 R14=c0217920 R15=c001dc94
> PSR=00000153 ---- A svc32
> ./run-emulator.sh: line 3: 22118 Aborted                 (core dumped)
> qemu-system-arm -M versatilepb -cpu arm1136 -nographic -no-reboot
> -kernel "zImage-armv6l" $WITH_HDC $WITH_HDB -hda "image-armv6l.sqf"
> -append "root=/dev/sda rw init=/sbin/init.sh panic=1
> PATH=$DISTCC_PATH_PREFIX/bin:/sbin console=ttyAMA0 HOST=armv6l
> $KERNEL_EXTRA" $QEMU_EXTRA -net nic,model=rtl8139 -net user
> 
> Instead system-image-arm4l works as expected.

armv4l is OABI, you might want armv4tl or armv5l.  (Those also currently
work, it's just armv6l that qemu barfs on.)

> Thank you very much!
> 
> Ciao,
> Alessio
> 

Rob

 1302004310.0

From dvstwo at gmail.com  Wed Apr  6 13:42:47 2011
From: dvstwo at gmail.com (David Seikel)
Date: Wed, 6 Apr 2011 23:42:47 +1000
Subject: [Aboriginal] Various things
In-Reply-To: <4D9C50A4.1090107@landley.net>
References: <BANLkTi=hFjHaBCF9-PjBHCAazBycqnmh_g@mail.gmail.com>
	<4D9C50A4.1090107@landley.net>
Message-ID: <20110406234247.35fad23a@U64-desktop>

On Wed, 06 Apr 2011 06:38:12 -0500 Rob Landley <rob at landley.net> wrote:

> I'm working on a gentoo-bootstrap which builds enough to run portage
> (and possibly catalyst).
> 
> I need to do a debian-bootstrap.hdc that builds enough prerequisites
> to make debootstrap work.  Once I've got that, Fedora has one called
> Koji:
> 
>   http://fedoraproject.org/wiki/Koji
> 
> And SuSE and Ubuntu and such have their own...

After this current project is done, I plan on going further (on my own
time though).  Don't hold your breath, but if you have not done it by
the time I get a round tuit, I'm likely to be working on either the
debian or ubuntu bootstraps.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110406/27b77987/attachment.sig>

From rob at landley.net  Wed Apr  6 11:38:12 2011
From: rob at landley.net (Rob Landley)
Date: Wed, 06 Apr 2011 06:38:12 -0500
Subject: [Aboriginal] Various things
In-Reply-To: <BANLkTi=hFjHaBCF9-PjBHCAazBycqnmh_g@mail.gmail.com>
References: <BANLkTi=hFjHaBCF9-PjBHCAazBycqnmh_g@mail.gmail.com>
Message-ID: <4D9C50A4.1090107@landley.net>

On 04/06/2011 04:05 AM, Alessio Igor Bogani wrote:
> Dear Mr. Landley,
> 
> Sorry to bother you directly again and for my very bad English.
> I hope that this is the last time I write privately (so I'll try to
> use ml instead).

I don't really mind, but people were wondering why the mailing list
isn't very highly trafficed...

> I have some considerations, thoughts and questions for you:
> 
> 1) You _should_ force people to use ml. Sound ironic that I said that
> in a private email but the mailing-list's archive vision is very
> distressing.

There used to be more archive but as I said I lost the old list when the
impactlinux.com domain went away, including the old subscriber list and
archives.  Then I didn't bother to set it back up for a few months, and
between those it lost rather a lot of momentum. :)

> 2) Aboriginal Linux is simple an horrible name (it sounds like the
> name of babies' toy)

The previous name was "Firmware Linux", which sounded like a bootloader.
 (And technically, it isn't firmware, so...)

> 3) Why have you chosen Mercurial instead of git?

I made the decision at OLS in 2006.  I tracked down and interviewed both
the git maintainer and the mercurial container, and saw the talks each
gave on their respective projects.  As I wrote in
http://landley.net/notes-2006.html at the time (July 23 entry):

  I'm converting the BusyBox repository to Mercurial.  (It was that or
  git, and it doesn't matter strongly which of the two because they
  work similarly enough that you can convert back and forth between
  them without losing the funky lateral branch history that makes 3-way
  merges work.)  I picked Mercurial because it's twice as fast, 1/10th
  as much code, and I understand the repository layout.

I keep meaning to set up a git mirror for the people who are more
familiar with that.  It's one of the many things on my todo list...

> 4) I hope that you'll replace busybox with toybox + useful packages
> (the first one is very a burden). The *box should do provide the
> simplest console interface on the system: other things should be
> delegated to specific project (like dropbear or lighthttpd).

This probably is a topic for the list. :)

I've been ambivalent about toybox for years:

  http://landley.net/notes-2009.html#24-08-2009
  http://lists.busybox.net/pipermail/busybox/2010-March/071783.html
  http://landley.net/notes-2009.html#18-06-2009
  http://landley.net/notes-2009.html#09-05-2009
  http://landley.net/notes-2009.html#26-02-2009

The point of Aboriginal Linux is to bootstrap the simplest possible
development environment for the target which is capable of rebuilding
itself under itself from source code.  At that point, you can build any
arbitrary system you want natively on the target.

I'm trying to separate what you build from how you build (which targets
you develop for should not influence which packages are available on the
target).  I'm very carefully NOT opening the can of worms of "being a
distro", there are already a dozen projects that do that quite well
thank you (and even more like buildroot and openembedded that do it
badly).  Debian has a repository of over 40,000 packages, with
dependnecy information and maintained build recipes.  I can't coompete
with that, and don't want to, nor do I want to tie you to chosing dpkg
instead of portage or rpm or ipkg or whatever repo you want to use, just
because you've chosen to target a given piece of hardware.

I did a long slide series on all this at one point:

  landley.net/aboriginal/downloads/presentation.pdf

It was designed to be a day long 8 hour tutorial, but everybody kept
wanting 1 hour talks with different subsets of the material instead...

> 5) What is your opinion about Aboriginal vs. Yocto?

As soon as a project gets Linux Foundation on it, it stops being
interesting.

  http://landley.net/notes-2010.html#27-10-2010
  http://landley.net/notes-2010.html#30-08-2010
  http://landley.net/notes-2010.html#18-07-2010
  http://landley.net/notes-2007.html#15-11-2007

> 6) I would want make Ubuntu packages for Aboriginal Linux. What do you
> think about?

Read the old release notes about "Automated native build control images":

  http://landley.net/aboriginal/#01-02-2011

Look at sources/control-images in the build scripts, and see also:

  http://landley.net/aboriginal/downloads/binaries/control-images/

I already did a Linux From Scratch bootstrap (although it's not fully
debugged on all targets due to uClibc 0.9.31's threading being wonky on
mips and powerpc and such, the builds behave differently if you run them
several times in a row.  That's a libc bug.  I'm hoping the NPTL stuff
in 0.9.32 will fix it.)

I'm working on a gentoo-bootstrap which builds enough to run portage
(and possibly catalyst).

I need to do a debian-bootstrap.hdc that builds enough prerequisites to
make debootstrap work.  Once I've got that, Fedora has one called Koji:

  http://fedoraproject.org/wiki/Koji

And SuSE and Ubuntu and such have their own...

But yes, that's a big todo item of mine.  Natively bootstrapping distros
in an architecture-independent manner.  Separate WHAT you build from HOW
you build.

I believe that presentation.pdf covers some of the design ideas later
on.  (It's a bit out of date, I need to update it.  The curse of doing
documentation, when you change the code or design you need to update the
docs...)

> These are only my personal (and stupids) opinions. After all I'm a
> very stupid and ignorant person.

Ignorant != stupid, and slow != stupid.  Can the false modesty please.
You have INTEREST in this area.  That's significant.  I'll take
passionate and slow over "smart but doesn't care" any day.

> Ciao,
> Alessio
> 
> P.s.
> I would want let you know that I have used your Aboriginal Linux to
> test my last patch:
> http://permalink.gmane.org/gmane.linux.kernel/1122916
> So thank you very *much* for Aboriginal Linux!

You're welcome.  Glad you like it. :)

Rob

 1302089892.0

From rlandley at parallels.com  Wed Apr  6 10:45:58 2011
From: rlandley at parallels.com (Rob Landley)
Date: Wed, 6 Apr 2011 05:45:58 -0500
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <20110405150643.GB11234@alice.local>
References: <20110331180847.1e70e680@U64-desktop>	<4D94771C.7070708@landley.net>	<20110402000443.2caa5c65@U64-desktop>	<4D96BEF5.5040401@parallels.com>	<20110405202405.4d285901@U64-desktop>
	<20110405150643.GB11234@alice.local>
Message-ID: <4D9C4466.9080505@parallels.com>

On 04/05/2011 10:06 AM, Alan Post wrote:
> On Tue, Apr 05, 2011 at 08:24:05PM +1000, David Seikel wrote:
>> BTW, is anyone else on this list, or did you loose them all and I'm
>> the first to find it?  Seems awfully quiet in here.
>>
> 
> I'm here; I have my own Aboriginal Linux project.  It is slow
> going, so I'm bursty writing the list.
> 
> -Alan

In the past week I got email from Alessio and John Bosch.

I finally managed to forward Alessio's email to the list after several
attempts.  That conversation continued off-list for a while and my
pending todo item from it is fix armv6l, which got broken by a kernel
upgrade and the _proper_ fix is probably to stop hacking the versatile
board but to actually use one of the other arm board emulations that
qemu now supports.  What I'm REALLY waiting for is for
http://people.canonical.com/~jk/dt-workshop/ to actually get all nicely
merged upstream.  Device tree support just went in for x86 in Linus's
tree, I'd call that a good sign...)

John Bosch was on the old list, he had a problem due to using too old a
qemu version, then had a native package build fail because #!/bin/sh
still points to hush last release, my bad, regression due to separating
the root filesystem build into simple-root-filesystem and
native-compiler combined by root-filesystem.sh.  (I fixed it in hg 1332
but haven't cut a release yet.)  I pointed him at the do_in_chroot
stuff, and I really need to move that into the base image and properly
document it.

The usual.  As much or more traffic off-list as on it.  I'm trying to
fix that, but since just about all the commits are by me, I can't really
blame 'em... :)

Rob

 1302086758.0

From bjorn.forsman at gmail.com  Thu Apr  7 06:33:57 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Thu, 7 Apr 2011 08:33:57 +0200
Subject: [Aboriginal] new user on the list (with questions)
Message-ID: <BANLkTi=idAQ5DGXATZtpHDTY1GURmsDmHA@mail.gmail.com>

Hi Rob,

My name is Bj?rn and I'm new on this list (have been lurking here for
a few weeks though). I have been doing embedded linux systems for
about three years. I'm a linux addict :-)

Most of the time I use Buildroot. But lately I've been wanting to use
a "real distro" instead of an "accidental distro" (I've been reading
your presentation.pdf and website :-) ). Unfortunately, it seems most
"real" distros don't really care about being able to easily bootstrap
themselves from source (as far as I can see). This is bad news for me,
because I think being able to build from source (with a specific
toolchain) is very important. I don't want to rely on binary stuff I
cannot rebuild.

So, I'm really glad to find the Aboriginal Linux project!

I have some questions:

* Where is the hg clone URL for Aboriginal Linux?
BTW, If you're considering setting up a git repo (I saw an email the
other day), maybe this helps: http://hg-git.github.com/

* Any chance Aboriginal Linux will support eglibc toolchains in the
future? Or any other libc than uClibc? Or is this a non-issue after
the next uClibc release as it will be possible to build other libc's
with the uClibc toolchain inside the emulator?

* What do you think about crosstool-NG? (It already has
uClibc/eglibc/glibc/newlib toolchains.) Could Aboriginal leverage the
work of crosstool-NG?

* Would you accept patches for an archlinux-bootstrap? It is currently
on my todo because Archlinux seems like a good "embedded" distro: it
has a small and efficient package manager written in C, works with
binary package repositories, the build scripts / package metadata are
simple shell scripts (easy to bootstrap) and the distro adheres to the
KISS principle. Any thoughts on that?

* It may seem that Aboriginal Linux has a very small community (little
ML traffic, all changes are done by you(?)), why do you think that is?

Thanks for writing Aboriginal Linux, I'm very excited about it :-)

Best regards,
Bj?rn Forsman

 1302158037.0

From rob at landley.net  Thu Apr  7 09:13:56 2011
From: rob at landley.net (Rob Landley)
Date: Thu, 07 Apr 2011 04:13:56 -0500
Subject: [Aboriginal] new user on the list (with questions)
In-Reply-To: <BANLkTi=idAQ5DGXATZtpHDTY1GURmsDmHA@mail.gmail.com>
References: <BANLkTi=idAQ5DGXATZtpHDTY1GURmsDmHA@mail.gmail.com>
Message-ID: <4D9D8054.6070306@landley.net>

On 04/07/2011 01:33 AM, Bj?rn Forsman wrote:
> Hi Rob,
> 
> My name is Bj?rn and I'm new on this list (have been lurking here for
> a few weeks though). I have been doing embedded linux systems for
> about three years. I'm a linux addict :-)

Hi Bjorn!  (Dunno how to do the 0 yet.)

> Most of the time I use Buildroot. But lately I've been wanting to use
> a "real distro" instead of an "accidental distro" (I've been reading
> your presentation.pdf and website :-) ). Unfortunately, it seems most
> "real" distros don't really care about being able to easily bootstrap
> themselves from source (as far as I can see). This is bad news for me,
> because I think being able to build from source (with a specific
> toolchain) is very important. I don't want to rely on binary stuff I
> cannot rebuild.

Linux From Scratch is the only "bootstrap from source" demo, and that
was easy-ish to automate (but still needs more testing, and more bugs
hammered out of the C library and configuration on non-x86 targets).

Gentoo  has an emphasis on building from source, but it expects you to
build Gentoo under Gentoo, not under another distro.  (So "build" not
quite equal "bootstrap".)  I have about half of a gentoo-bootstrap done,
which has been starved for attention for a while now.

Lots of distros have a root filesystem creator, something that populates
a chroot with a new root filesystem.  In debian the package is
"debootstrap", in gentoo it's "catalyst", in Red hat there's "Koji", and
so on.  (I have the SuSE and ubuntu ones written down around here
somewhere.  Oh, and Red Hat has a wrapper around Koji, but I have yet to
quite work out what it does.)

Unfortunately, these root filesystem creators generally work from their
binary repositories, not their source repositories.  They install a set
of prebuilt binary packages into the new chroot, until they have
something you can chroot into and run (plus dev tools until it's capable
of rebuilding itself, ad nauseum).

So what I've been doing is:

A) Find their package management system.  portage, rpm, dpkg, etc.

B) Figure out what's involved in building packages through said package
management system.  Red hat has "source rpms", gentoo has emerge, etc.
Sometime's there's reasonable documentation (such as
http://www.debian.org/doc/maint-guide/start.en.html#needprogs ) and
sometimes you just have to try it and see why it fails.

C) Try to cobble together a root filesystem in which their build runs,
and document the steps in a reproducible build script.

D) Build and test lots of packages to shake the bugs out.  (Having a
known working Linux From Scratch environment helps here, once you fix a
bug in uClibc and such it should be fixed for all distros...)

I've actually been meaning to take openembedded and buildroot and such
and treat them as small distros: try to chop off their "what you build"
and "how you build" bits to convince them to natively build their
packages on an unknown target.  I.E. treat buildroot like koji or
catalyst: something that populates a chroot using the host's native
toolchain.

Although technically those all want to install a new toolchain into the
chroot.  Which is sort of ok, except:

A) providing a working toolchain and libc on a given host isn't trivial.
 (The point of native compiling is you've already _got_ that, not that
it was easy to achieve, or to reproduce using different package versions
in who knows what configuration).

B) It breaks the distcc trick.  Not fatal, but very convenient if you
can keep that going.

> So, I'm really glad to find the Aboriginal Linux project!
> 
> I have some questions:
> 
> * Where is the hg clone URL for Aboriginal Linux?

hg clone http://landley.net/hg/aboriginal

> BTW, If you're considering setting up a git repo (I saw an email the
> other day), maybe this helps: http://hg-git.github.com/

A mirror.  I'd still apply new changes to the hg repo, but I could
cherry pick them out of a git repo if somebody else had one.  (Not sure
how to convince it the changes were the same for merge purposes, depends
on the clone tool.)

Looks like a nice tool.  I've bookmarked it, and will try to make time
to look at it properly this weekend.  Thanks.

> * Any chance Aboriginal Linux will support eglibc toolchains in the
> future? Or any other libc than uClibc? Or is this a non-issue after
> the next uClibc release as it will be possible to build other libc's
> with the uClibc toolchain inside the emulator?

That last one is the design goal.  I don't want to cross compile more
than one setup (eglibc is basically a reskinned glibc rather than a
separate project, and building glibc requires perl), but once uClibc has
TLS then glibc should build under it.  And THAT I want to have a
control-image to build.  (I'll might update the Linux From Scratch thing
to actually build the full toolchain, especially if binutils and gcc
will autodetect the host properly.  Yeah, I know I said that's a pain
with catalyst and such, but _those_ things are full of baked-in
assumptions that the distro they're bootstrapping is only ever built
under itself.  LFS has the whole chapter 5 airlock step.)

> * What do you think about crosstool-NG? (It already has
> uClibc/eglibc/glibc/newlib toolchains.) Could Aboriginal leverage the
> work of crosstool-NG?

Yup.

My build is a series of orthogonal stages, each designed to be replaced.
 The simple-cross-compiler, cross-compiler, and native-compiler stages
should all be disposable if you've got something else.  (You can just
untar it into the build directory, fix up the names, and run the next
stage.  You may have to touch the tarball name for build.sh to accept
that it's already there, that thing does dependency checking by seeing
if the previous stage's tarball is there or not.  Or just skip build.sh
and run the stages yourself, it's intentionally a trivial wrapper.)

> * Would you accept patches for an archlinux-bootstrap?

Yes please. :)

I spoke with the guy running the arch linux booth at scale, but
unfortunately he was not (remotely) an expert.  Couldn't really answer
any of my questions...

> It is currently
> on my todo because Archlinux seems like a good "embedded" distro: it
> has a small and efficient package manager written in C, works with
> binary package repositories, the build scripts / package metadata are
> simple shell scripts (easy to bootstrap) and the distro adheres to the
> KISS principle. Any thoughts on that?

It's already on my todo list, but several places down.

At the top of my todo list is finishing gentoo-boostrap because A) I've
already done half the work for that (and 2/3 of the research), B) it
builds from source already.

After that probably Debian/Ubuntu, becuase thats what I use on my laptop
and Ubuntu is the most popular Linux distro with somewhere around 40%
market share.  (And considerably higher among developer workstations.)

Red Hat had 50% market share among all Linux installs back when Red Hat
9 shipped, but then they figured out how to steal Sun's air supply.
Solaris' core market was large corporate and government procurement
contracts, which "protected" themselves by capping a contractor's
allowable profit margin at a percentage of the cost of materials.  So
the bidders specced the most expensive materials they could: instead of
a $30 Red Hat seat they specced a $5000 Solaris seat, because they'd
make $500 instead of $3.  When Red Hat figured this out and came out
with "Red Hat Enterprise" (I.E. "Pointy Hair Linux") they suddenly had
$100 million in annual orders from the fortune 500 (which technically
preferred Linux but wouldn't leave money on the table).  This WRENCHED
Red Hat's attention away from the desktop market (where they were making
maybe $10-$20 million annually), they discontinued it, renamed Rawhide
to "Fedora" (because their enterprise versions still needed R&D)...

Ahem.  Digression. :)

> * It may seem that Aboriginal Linux has a very small community (little
> ML traffic, all changes are done by you(?)), why do you think that is?

Really bad habits on my part?

  http://landley.net/aboriginal/history.html

This project has been one of my main hobbies for most of a decade now,
but until recently it was more exporatory/experimental than practical.
It's been relaunched from scratch three times (as I figured out "that's
not how I should be doing this"), and spun off a lot of tangents that
distracted me (yes, becoming busybox maintainer was a _tangent_ from
this project).

The problem for many years was I didn't have a clear idea what I was
trying to do, and didn't know how to do it anyway.  I was reacting
AGAINST the horrible bloated crappy GNU software, and the fact that my
first Linux machine ran great in 8 megs of ram but five years later you
couldn't even BOOT in 16 megs.  That was wrong and I wanted to stop it,
I just wasn't sure _how_.

As an example of one such shifting goal: after Red Hat got out of the
distro business (and Fedora Core 2 was such a disaster I declared it
unusable and started looking for other distros to use), I ran knoppix on
my laptop for a while.  Circa 2003 it was the ONLY distro even _trying_
for Linux on the desktop.  (I'm serious: I couldn't find anything else.
 I looked.  SuSE was going through bankruptcy, the Slackware guy had
some sort of terminal disease, Debian's infighting had completely
paralyzed the project, the gentoo people REFUSED to have anything to do
with installers because installing via extensive howto was considered a
rite of passage...  Lots of once-big distros like TurboLinux, Caldera,
Mandrake, and Xandros completely collapsed...  Ubuntu didn't ship its
first release until October 2004, and that first release was crap.)

I thought knoppix was great, but it used glibc and the gnu tools, which
right there took up maybe 10% of the whole CD.  Wouldn't a bootable CD
based on uClibc and busybox make more sense, and save more space for
other stuff?  (That was my vague end-goal in the 2003-2005 range, before
QEMU happened and I discovered cross compiling.)

It's hard to recruit other people to help you out with _design_ issues,
and with project goals.  I knew I wanted to make busybox into a
replacement for the gnu tools, and uClibc a replacement for gcc, and get
the FSF's "gnu/gnu/gnu/dammit we take credit for everything" hands off
of Linux.  But although that was an invariant all along, it was also
incidental.  I was always reaching for some goal beyond that.  (Such as
the current "make cross compiling go away", "make diverse Linux
platforms work the same", "perform automated cross-platform regression
testing", and so on.  The presentation listed several of them.  You'll
notice a theme here: what should we NOT be doing.  How can I demonstrate
that we don't NEED to do this?  That embedded development doesn't need
to mix cross compiling with package selection, for example...)

The project's current set of design goals tightened up around 2008 or so
(writing up everything I could think of to make the presentation.pdf
helped a lot, documenting stuff really does clarify your thoughts), and
september 2010 I got close enough to them to cut a 1.0 release that more
or less does that, from the right angle, with a tailwind.

The project's first short-lived mailing list wasn't until 2007 or so,
and this one that just restarted is its... third?  Fourth?  I never
really encouraged people to use the list instead of emailing me directly
so the majority of the traffic was always people emailing me instead of
people emailing the list.  (Horribly bad habit, I know.)

My blog (http://landley.net and http://landley.livejournal.com before
that) were the big "what have I been doing on this project" things
(updated daily for several years, http://landley.net/notes-2008.html has
347 daily entries out of 366 days in that year), but the blogs on
landley.net have no way to comment on them.  (More convenient for me,
less convenient for others.)

I became BusyBox maintainer for a year and change because of this
project (which essentially shut down work on FWL/Aboriginal because I
didn't have time).  When I started trying to substitute busybox for the
gnu tools it was a crazy thing to do that didn't remotely work: I spent
years _making_ it work, starting by rewriting "sed" until I could
substitute the busybox sed for gnu sed in the Linux From Scratch build
and the result actually worked right for all packages.  Then it was
replacing one command at a time and fixing each one.  I left busybox
because of bad blood between me and Bruce Fscking Perens, which
prevented me from upgrading busybox for a couple years and instead
working to reimplement it ala http://landley.net/code/toybox (I got over
it in 2008, commit 358, but that was another year or so of effort
diverted from working on this project).

There were other distractions.  I spent a year or so forking and
developing tinycc instead of focusing on this.  (I stopped when I
stimulated the original tinycc project into coming out of retirement,
Fabrice handed off maintainership to a windows developer and it became a
primarily windows project, not very interesting but still not something
I want to compete with.)  The uClibc project stalled horribly for years
and I had to do some political maneuvering to get that unstuck last year...

Anyway, I decided to get serious about it in 2008, and even start an
embedded consulting company so I could spend more time working on it and
less on other things.  That was a bad idea: between the recession and
running a company being a huge time sink, Mark and I folded that the
following year (and several months later when Mark wiped the server
running the domain I still hadn't migrated the project off yet; oops.
Lost the old mailing list.)

These days my day job is doing Linux kernel work for parallels, for the
past few months fighting with the NFS code to make it work in
containers.  Which is interting but again takes time away from working
on this.

Now that I have the 1.0 release out, the work is:

A) Regression testing against each new package release.  (Every time I
upgrade the kernel, or busybox, or uClibc, or qemu, or anything
really... it breaks some target somewhere.)

B) Making more targets work.  (Including fixing things like sh4, mips64,
and sparc that sort of work, or used to work, but can't build Linux From
Scratch.)

C) Cleanly separating the orthogonal layers so you _can_ swap in other
toolchains and such, or use this system-image script to package up
somebody else's root filesystem, or...

D) Documenting everything properly.  (I keep meaning to do a series of
podcasts explaining stuff, I haven't made the time and haven't got
decent recording equipment anyway.)

E) Then next big project on top of this: system images and bootstraping
distros.  (Which probably _is_ a separate project, and may need its own
repository.)

By the way, I'm currently stuck on the last gplv2 release of binutils
and gcc the same way I was stuck on busybox 1.2.2.  (I'm never going to
contribute to any GPLv3 licensed project unless paid, and I'm not
comfortable shipping binaries of that stuff.)  Unfortunately that means
I can't do armv7 and the microblaze targets.  I'm subscribed to the pcc
mailing list and I check in on llvm/clang from time to time, and I
really hope one of those unseats gcc.  (That's why I was working on
tinycc, but Grischka's permanently rendered that project irrelevant.)

Using crosstool-ng would be a great workaround for that.  They do
toolchains so I don't have to.  And if ALL they do is toolchains: great,
leverage it.  (But I need cross and native, and I'm picky about 'em, and
I still need to build my own C library because uClibc ain't fully cooked
yet, and ccwrap.c is a better approach than trying to get gcc's path
logic to _EVER_ work...)

> Thanks for writing Aboriginal Linux, I'm very excited about it :-)
> 
> Best regards,
> Bj?rn Forsman

Woot.  Thanks.  Let me know what I can do to help. :)

Rob

 1302167636.0

From dvstwo at gmail.com  Thu Apr  7 10:07:47 2011
From: dvstwo at gmail.com (David Seikel)
Date: Thu, 7 Apr 2011 20:07:47 +1000
Subject: [Aboriginal] new user on the list (with questions)
In-Reply-To: <4D9D8054.6070306@landley.net>
References: <BANLkTi=idAQ5DGXATZtpHDTY1GURmsDmHA@mail.gmail.com>
	<4D9D8054.6070306@landley.net>
Message-ID: <20110407200747.75271386@U64-desktop>

On Thu, 07 Apr 2011 04:13:56 -0500 Rob Landley <rob at landley.net> wrote:

> On 04/07/2011 01:33 AM, Bj?rn Forsman wrote:
> > Hi Rob,
> > 
> > My name is Bj?rn and I'm new on this list (have been lurking here
> > for a few weeks though). I have been doing embedded linux systems
> > for about three years. I'm a linux addict :-)
> 
> Hi Bjorn!  (Dunno how to do the 0 yet.)

Copy and paste it from Bj?rn's email, like I just did?  Note that your
email proggy got it right in the "> On 04/07/2011 01:33 AM, Bj?rn
Forsman wrote:" part.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110407/95fa7134/attachment.sig>

From dvstwo at gmail.com  Thu Apr  7 10:17:30 2011
From: dvstwo at gmail.com (David Seikel)
Date: Thu, 7 Apr 2011 20:17:30 +1000
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
In-Reply-To: <4D9B0256.2000502@parallels.com>
References: <4D9B0256.2000502@parallels.com>
Message-ID: <20110407201730.06d0f6ab@U64-desktop>

On Tue, 5 Apr 2011 06:51:50 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> Retry sending it form my work address?  (Yes, gmail's spam filtering
> is so broken I have to send this through an exchange server to get it
> out. I'm going to have to set up my own mail server again, aren't I?)

I've been using a simple hack to work around gmails failings with
mailing lists for years, since google say it's a feature, not a bug.

You might notice I have a read only and a write only account on your
list.  I write an email to the list using this account, but tell your
mailing list to NOT send this account any of the list email.  This is
the write only account.  The read only acount is the one that actually
has your mailing list configured to send me the mails.  Since none of
those mails where written by the read only account, gmail lets it
through.

It's slightly annoying that those people replying to my posts that send
me CCs as well (in itself annoying when I'm ON THE DAMN LIST), don't
have the duplicate emails filtered out by the mailing lists "don't send
me the damn CCs" feature.  Coz I get the emails from the read only
account, and it does not know about the write only account to know to
filter them.  Oh well, rather get too much than not enough.

Other people have told google in no uncertain terms that filtering out
your own emails is a bug, NOT a feature for many years.  sigh

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110407/5ad6c303/attachment.sig>

From rlandley at parallels.com  Thu Apr  7 10:22:17 2011
From: rlandley at parallels.com (Rob Landley)
Date: Thu, 7 Apr 2011 05:22:17 -0500
Subject: [Aboriginal] new user on the list (with questions)
In-Reply-To: <20110407200747.75271386@U64-desktop>
References: <BANLkTi=idAQ5DGXATZtpHDTY1GURmsDmHA@mail.gmail.com>	<4D9D8054.6070306@landley.net>
	<20110407200747.75271386@U64-desktop>
Message-ID: <4D9D9059.7010706@parallels.com>

On 04/07/2011 05:07 AM, David Seikel wrote:
> On Thu, 07 Apr 2011 04:13:56 -0500 Rob Landley <rob at landley.net> wrote:
> 
>> On 04/07/2011 01:33 AM, Bj?rn Forsman wrote:
>>> Hi Rob,
>>>
>>> My name is Bj?rn and I'm new on this list (have been lurking here
>>> for a few weeks though). I have been doing embedded linux systems
>>> for about three years. I'm a linux addict :-)
>>
>> Hi Bjorn!  (Dunno how to do the 0 yet.)
> 
> Copy and paste it from Bj?rn's email, like I just did?  Note that your
> email proggy got it right in the "> On 04/07/2011 01:33 AM, Bj?rn
> Forsman wrote:" part.

Yes, but that's cheating.  I meant I dunno how to type it from the
keyboard yet.  (I just looked it up: ctrl-shit-u followed by 00f8 then
hit space, which I'm sure I'll remember for _minutes_ yet: ?)

In theory there's a "composition table", in practice it's not working
for me in xfce.

In Bj?rn's case I suspect he has a different keyboard layout than I do. :)

Rob

 1302171737.0

From rlandley at parallels.com  Thu Apr  7 10:55:01 2011
From: rlandley at parallels.com (Rob Landley)
Date: Thu, 7 Apr 2011 05:55:01 -0500
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
In-Reply-To: <20110407201730.06d0f6ab@U64-desktop>
References: <4D9B0256.2000502@parallels.com>
	<20110407201730.06d0f6ab@U64-desktop>
Message-ID: <4D9D9805.7000001@parallels.com>

On 04/07/2011 05:17 AM, David Seikel wrote:
> On Tue, 5 Apr 2011 06:51:50 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
>> Retry sending it form my work address?  (Yes, gmail's spam filtering
>> is so broken I have to send this through an exchange server to get it
>> out. I'm going to have to set up my own mail server again, aren't I?)
> 
> I've been using a simple hack to work around gmails failings with
> mailing lists for years, since google say it's a feature, not a bug.

They're wrong.

> You might notice I have a read only and a write only account on your
> list.

Yeah, I'm doing that.  It's still not working around _this_ bug.  Gmail
is saying "you already sent that message to somebody else" and filtering
it out when I send it to a different (overlapping) group of people.

> I write an email to the list using this account, but tell your
> mailing list to NOT send this account any of the list email.  This is
> the write only account.  The read only acount is the one that actually
> has your mailing list configured to send me the mails.  Since none of
> those mails where written by the read only account, gmail lets it
> through.
> 
> It's slightly annoying that those people replying to my posts that send
> me CCs as well (in itself annoying when I'm ON THE DAMN LIST), don't
> have the duplicate emails filtered out by the mailing lists "don't send
> me the damn CCs" feature.  Coz I get the emails from the read only
> account, and it does not know about the write only account to know to
> filter them.  Oh well, rather get too much than not enough.

I'm on a lot of high traffic lists where if they don't cc: me I won't
notice for a week.

> Other people have told google in no uncertain terms that filtering out
> your own emails is a bug, NOT a feature for many years.  sigh

Yes, as Google Buzz and Wave and Orkut and such show, they're never wrong:

  http://landley.livejournal.com/48903.html

I need ot set up my own mail server.  I already poked Chris DiBona about
it on twitter, which is pretty much the only way I've ever found of
getting Google's attention about anything.

Rob

 1302173701.0

From onefang at gmail.com  Thu Apr  7 11:18:23 2011
From: onefang at gmail.com (David Seikel)
Date: Thu, 7 Apr 2011 21:18:23 +1000
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
In-Reply-To: <4D9D9805.7000001@parallels.com>
References: <4D9B0256.2000502@parallels.com>
	<20110407201730.06d0f6ab@U64-desktop>
	<4D9D9805.7000001@parallels.com>
Message-ID: <20110407211823.714eda02@U64-desktop>

On Thu, 7 Apr 2011 05:55:01 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> On 04/07/2011 05:17 AM, David Seikel wrote:
> > On Tue, 5 Apr 2011 06:51:50 -0500 Rob Landley
> > <rlandley at parallels.com> wrote:
> > 
> >> Retry sending it form my work address?  (Yes, gmail's spam
> >> filtering is so broken I have to send this through an exchange
> >> server to get it out. I'm going to have to set up my own mail
> >> server again, aren't I?)
> > 
> > I've been using a simple hack to work around gmails failings with
> > mailing lists for years, since google say it's a feature, not a bug.
> 
> They're wrong.
> 
> > You might notice I have a read only and a write only account on your
> > list.
> 
> Yeah, I'm doing that.  It's still not working around _this_ bug.
> Gmail is saying "you already sent that message to somebody else" and
> filtering it out when I send it to a different (overlapping) group of
> people.

Naturally the moment I comment on that in this list, it stops working.
I'm not getting my emails through the read only acount. lol

/me pokes at his accounts.

Does not seem to be anything wrong there.  Oh, maybe I'm sending from
the read only account now?  How did that happen?

> > Other people have told google in no uncertain terms that filtering
> > out your own emails is a bug, NOT a feature for many years.  sigh
> 
> Yes, as Google Buzz and Wave and Orkut and such show, they're never
> wrong:
> 
>   http://landley.livejournal.com/48903.html

500 Internal Server Error


-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110407/09db7388/attachment.sig>

From bjorn.forsman at gmail.com  Thu Apr  7 11:29:13 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Thu, 7 Apr 2011 13:29:13 +0200
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
In-Reply-To: <20110407201730.06d0f6ab@U64-desktop>
References: <4D9B0256.2000502@parallels.com>
	<20110407201730.06d0f6ab@U64-desktop>
Message-ID: <BANLkTi=ccarVx+qyhd5f_28xeQjysjCWDw@mail.gmail.com>

On 7 April 2011 12:17, David Seikel <dvstwo at gmail.com> wrote:
> On Tue, 5 Apr 2011 06:51:50 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
>
>> Retry sending it form my work address? ?(Yes, gmail's spam filtering
>> is so broken I have to send this through an exchange server to get it
>> out. I'm going to have to set up my own mail server again, aren't I?)
>
> I've been using a simple hack to work around gmails failings with
> mailing lists for years, since google say it's a feature, not a bug.

How come my gmail account doesn't have these problems?

Whenever I subscribe to a ML I setup a new filter in gmail that
automatically lables and archives the emails. I've never had any
issues.

Best regards,
Bj?rn Forsman

 1302175753.0

From bjorn.forsman at gmail.com  Thu Apr  7 11:32:05 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Thu, 7 Apr 2011 13:32:05 +0200
Subject: [Aboriginal] new user on the list (with questions)
In-Reply-To: <4D9D9059.7010706@parallels.com>
References: <BANLkTi=idAQ5DGXATZtpHDTY1GURmsDmHA@mail.gmail.com>
	<4D9D8054.6070306@landley.net>
	<20110407200747.75271386@U64-desktop>
	<4D9D9059.7010706@parallels.com>
Message-ID: <BANLkTikcS9Jq+Uv5Jw4i4YHnXpr3d3x6gw@mail.gmail.com>

On 7 April 2011 12:22, Rob Landley <rlandley at parallels.com> wrote:
> On 04/07/2011 05:07 AM, David Seikel wrote:
>> On Thu, 07 Apr 2011 04:13:56 -0500 Rob Landley <rob at landley.net> wrote:
>>
>>> On 04/07/2011 01:33 AM, Bj?rn Forsman wrote:
>>>> Hi Rob,
>>>>
>>>> My name is Bj?rn and I'm new on this list (have been lurking here
>>>> for a few weeks though). I have been doing embedded linux systems
>>>> for about three years. I'm a linux addict :-)
>>>
>>> Hi Bjorn! ?(Dunno how to do the 0 yet.)
>>
>> Copy and paste it from Bj?rn's email, like I just did? ?Note that your
>> email proggy got it right in the "> On 04/07/2011 01:33 AM, Bj?rn
>> Forsman wrote:" part.
>
> Yes, but that's cheating. ?I meant I dunno how to type it from the
> keyboard yet. ?(I just looked it up: ctrl-shit-u followed by 00f8 then
> hit space, which I'm sure I'll remember for _minutes_ yet: ?)
>
> In theory there's a "composition table", in practice it's not working
> for me in xfce.
>
> In Bj?rn's case I suspect he has a different keyboard layout than I do. :)

Correct :-)

Don't worry about the '?' though, I can live with an 'o' :-)

 1302175925.0

From onefang at gmail.com  Thu Apr  7 11:45:23 2011
From: onefang at gmail.com (David Seikel)
Date: Thu, 7 Apr 2011 21:45:23 +1000
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
In-Reply-To: <BANLkTi=ccarVx+qyhd5f_28xeQjysjCWDw@mail.gmail.com>
References: <4D9B0256.2000502@parallels.com>
	<20110407201730.06d0f6ab@U64-desktop>
	<BANLkTi=ccarVx+qyhd5f_28xeQjysjCWDw@mail.gmail.com>
Message-ID: <20110407214523.69e05f0f@U64-desktop>

On Thu, 7 Apr 2011 13:29:13 +0200 Bj?rn Forsman
<bjorn.forsman at gmail.com> wrote:

> On 7 April 2011 12:17, David Seikel <dvstwo at gmail.com> wrote:
> > On Tue, 5 Apr 2011 06:51:50 -0500 Rob Landley
> > <rlandley at parallels.com> wrote:
> >
> >> Retry sending it form my work address? ?(Yes, gmail's spam
> >> filtering is so broken I have to send this through an exchange
> >> server to get it out. I'm going to have to set up my own mail
> >> server again, aren't I?)
> >
> > I've been using a simple hack to work around gmails failings with
> > mailing lists for years, since google say it's a feature, not a bug.
> 
> How come my gmail account doesn't have these problems?
> 
> Whenever I subscribe to a ML I setup a new filter in gmail that
> automatically lables and archives the emails. I've never had any
> issues.

Perhaps you are using the web interface, and we are using POP to a real
email client?  The problem I have is the automatic archiving of the
emails I send, and once archived, POP can't access them, so I never see
them.  I LIKE seeing my emails coming back from a mailing list.  Tells
me that the "best effort, but no guarantees" email system is actually
working.  Also helps with threading at this end.

Might be worth while seeing if IMAP access to gmail lets you look at
the archives.  But still that's gmail doing things with my email that I
don't want.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110407/a0ad7019/attachment.sig>

From onefang at gmail.com  Thu Apr  7 11:47:01 2011
From: onefang at gmail.com (David Seikel)
Date: Thu, 7 Apr 2011 21:47:01 +1000
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
In-Reply-To: <20110407211823.714eda02@U64-desktop>
References: <4D9B0256.2000502@parallels.com>
	<20110407201730.06d0f6ab@U64-desktop>
	<4D9D9805.7000001@parallels.com>
	<20110407211823.714eda02@U64-desktop>
Message-ID: <20110407214701.37ad3f73@U64-desktop>

On Thu, 7 Apr 2011 21:18:23 +1000 David Seikel <onefang at gmail.com>
wrote:

> On Thu, 7 Apr 2011 05:55:01 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
> > On 04/07/2011 05:17 AM, David Seikel wrote:
> > > On Tue, 5 Apr 2011 06:51:50 -0500 Rob Landley
> > > <rlandley at parallels.com> wrote:
> > > 
> > >> Retry sending it form my work address?  (Yes, gmail's spam
> > >> filtering is so broken I have to send this through an exchange
> > >> server to get it out. I'm going to have to set up my own mail
> > >> server again, aren't I?)
> > > 
> > > I've been using a simple hack to work around gmails failings with
> > > mailing lists for years, since google say it's a feature, not a
> > > bug.
> > 
> > They're wrong.
> > 
> > > You might notice I have a read only and a write only account on
> > > your list.
> > 
> > Yeah, I'm doing that.  It's still not working around _this_ bug.
> > Gmail is saying "you already sent that message to somebody else" and
> > filtering it out when I send it to a different (overlapping) group
> > of people.
> 
> Naturally the moment I comment on that in this list, it stops working.
> I'm not getting my emails through the read only acount. lol
> 
> /me pokes at his accounts.
> 
> Does not seem to be anything wrong there.  Oh, maybe I'm sending from
> the read only account now?  How did that happen?

Oops, WAS sending from my read only account.  Silly me.  Al fixed now.

After breakfast I'll get back to actual Aboriginal Linux work.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110407/dc5b9a15/attachment.sig>

From onefang at gmail.com  Thu Apr  7 13:21:34 2011
From: onefang at gmail.com (David Seikel)
Date: Thu, 7 Apr 2011 23:21:34 +1000
Subject: [Aboriginal] undefined reference to `finite'
In-Reply-To: <4D9AFE66.7040906@parallels.com>
References: <20110405201205.5542b8c9@U64-desktop>
	<4D9AFE66.7040906@parallels.com>
Message-ID: <20110407232134.09936acb@U64-desktop>

On Tue, 5 Apr 2011 06:35:02 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> On 04/05/2011 05:12 AM, David Seikel wrote:
> > I'm very tired, and slightly ill, and I can't figure this out.
> > 
> > One of the packages from the UI library I want to use for this
> > project uses a function called finite(double).  As far as I can
> > tell, uClibc should supply that function, since we have
> > DO_C99_MATH=y in its baseconfig. It's not being found though.
> > 
> > It's all buried in a forest of #ifdef's that my tired mind is in no
> > condition to sort out.
> > 
> > I'm only two packages away for having all my needed libraries
> > installed before I can start cutting code.  Time to sleep.
> 
> Hmmm...  Posibly a question for the uClibc mailing list?
> 
> You didn't say which target you were using, so I'll poke at x86_64:

I'm sticking with doing my builds for 486, since it's still possible
that those tiny 486 based boards I looked at last year may be used.
If they are not used, then 486 code will at least work fine on whatever
we move to.  Any board we move to will be a lot faster anyway, and I've
already mentioned that speed is not important for this project.

It appears that finite() is obsolete, and replaced with an isfinite()
macro.  A few things I found in my research pointed to that, including
the man pages on Ubuntu, but the clincher was this -

http://www.mail-archive.com/uclibc at uclibc.org/msg03404.html

I'll poke the UI library development list about changing that, though
the source I'm using is a stable release tarball, but they are thinking
of cutting another release soon.  Wont be a major change, I think they
only use finite() once.

Think I'll just stick a sed command in the build script for that
library for now.

> By the way, have you read this yet?
> 
>   http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html
> 
> One of the best quick and dirty "get familiar with elf file format"
> things I've seen...

Nope, but I'll look at it later.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110407/75e275d3/attachment.sig>

From bjorn.forsman at gmail.com  Thu Apr  7 13:29:36 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Thu, 7 Apr 2011 15:29:36 +0200
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
In-Reply-To: <20110407214523.69e05f0f@U64-desktop>
References: <4D9B0256.2000502@parallels.com>
	<20110407201730.06d0f6ab@U64-desktop>
	<BANLkTi=ccarVx+qyhd5f_28xeQjysjCWDw@mail.gmail.com>
	<20110407214523.69e05f0f@U64-desktop>
Message-ID: <BANLkTi=AyWfRbdu5ztTqoZ48ON-X8Es5uA@mail.gmail.com>

On 7 April 2011 13:45, David Seikel <onefang at gmail.com> wrote:
> Perhaps you are using the web interface, and we are using POP to a real
> email client?

Yes.

 1302182976.0

From onefang at gmail.com  Thu Apr  7 15:17:19 2011
From: onefang at gmail.com (David Seikel)
Date: Fri, 8 Apr 2011 01:17:19 +1000
Subject: [Aboriginal] undefined reference to `finite'
In-Reply-To: <20110407232134.09936acb@U64-desktop>
References: <20110405201205.5542b8c9@U64-desktop>
	<4D9AFE66.7040906@parallels.com>
	<20110407232134.09936acb@U64-desktop>
Message-ID: <20110408011719.52c516d1@U64-desktop>

On Thu, 7 Apr 2011 23:21:34 +1000 David Seikel <onefang at gmail.com>
wrote:

> On Tue, 5 Apr 2011 06:35:02 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
> > On 04/05/2011 05:12 AM, David Seikel wrote:
> > > I'm very tired, and slightly ill, and I can't figure this out.
> > > 
> > > One of the packages from the UI library I want to use for this
> > > project uses a function called finite(double).  As far as I can
> > > tell, uClibc should supply that function, since we have
> > > DO_C99_MATH=y in its baseconfig. It's not being found though.
> > > 
> > > It's all buried in a forest of #ifdef's that my tired mind is in
> > > no condition to sort out.
> > > 
> > > I'm only two packages away for having all my needed libraries
> > > installed before I can start cutting code.  Time to sleep.
> > 
> > Hmmm...  Posibly a question for the uClibc mailing list?
> > 
> > You didn't say which target you were using, so I'll poke at x86_64:
> 
> I'm sticking with doing my builds for 486, since it's still possible
> that those tiny 486 based boards I looked at last year may be used.
> If they are not used, then 486 code will at least work fine on
> whatever we move to.  Any board we move to will be a lot faster
> anyway, and I've already mentioned that speed is not important for
> this project.
> 
> It appears that finite() is obsolete, and replaced with an isfinite()
> macro.  A few things I found in my research pointed to that, including
> the man pages on Ubuntu, but the clincher was this -
> 
> http://www.mail-archive.com/uclibc at uclibc.org/msg03404.html
> 
> I'll poke the UI library development list about changing that, though
> the source I'm using is a stable release tarball, but they are
> thinking of cutting another release soon.  Wont be a major change, I
> think they only use finite() once.
> 
> Think I'll just stick a sed command in the build script for that
> library for now.

That did the trick.  All the libraries and development tools I think
I'll need are now compiled.  YAY!!

Got it all in a modified version of lfs-bootstrap, so that's the entire
thing automated.  The actual application itself can be handled by just
one more small build script added to the end of that bootstrap.  It was
all very easy thanks to Aboriginal Linux.  I'm ahead of schedule.

Next steps - generate an actual bootable disk image that I can throw
at real hardware.  Clean up some loose ends I kinda glossed over in the
effort to get it all done.  Then... cut code.  B-)

> > By the way, have you read this yet?
> > 
> >   http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html
> > 
> > One of the best quick and dirty "get familiar with elf file format"
> > things I've seen...
> 
> Nope, but I'll look at it later.

Very interesting read.  Would have been magical if he had been able to
squeeze a little more and get the total executable size down to 42
bytes.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110408/9a11e581/attachment.sig>

From bjorn.forsman at gmail.com  Thu Apr  7 15:59:29 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Thu, 7 Apr 2011 17:59:29 +0200
Subject: [Aboriginal] new user on the list (with questions)
In-Reply-To: <4D9D8054.6070306@landley.net>
References: <BANLkTi=idAQ5DGXATZtpHDTY1GURmsDmHA@mail.gmail.com>
	<4D9D8054.6070306@landley.net>
Message-ID: <BANLkTinguSKUaA5wGWRwhrbktK1pM0sU4A@mail.gmail.com>

Hi Rob,

Thanks for that long (and quick!) reply! Please don't think I'm rude
just because my reply won't be nearly as long :-)

2011/4/7 Rob Landley <rob at landley.net>:
> On 04/07/2011 01:33 AM, Bj?rn Forsman wrote:
>> Hi Rob,
[snip]
> I've actually been meaning to take openembedded and buildroot and such
> and treat them as small distros: try to chop off their "what you build"
> and "how you build" bits to convince them to natively build their
> packages on an unknown target. ?I.E. treat buildroot like koji or
> catalyst: something that populates a chroot using the host's native
> toolchain.

You mean creating buildroot and OE control-images? (I wasn't expecting
that after the "accidental distro" thing.)

>> * Where is the hg clone URL for Aboriginal Linux?
>
> hg clone http://landley.net/hg/aboriginal

Ah, so it's the same as the webUI! I was so close to trying that :-)

> At the top of my todo list is finishing gentoo-boostrap because A) I've
> already done half the work for that (and 2/3 of the research), B) it
> builds from source already.

What's missing in gentoo-bootstrap? (I haven't tried it yet.)

It's really nice that Gentoo has one package metadata repository for
all ARCHs (they do right?), but having a package manager written in
python feels bloated for embedded systems.

Archlinux is pretty easy to build from source also (see Arch Build
System). But upstream Archlinux doesn't care about other architectures
than x86/x86_64 though, so that's a minus.

> After that probably Debian/Ubuntu, becuase thats what I use on my laptop
> and Ubuntu is the most popular Linux distro with somewhere around 40%
> market share. ?(And considerably higher among developer workstations.)

Great! I'm glad you're planning more bootstraps.

> By the way, I'm currently stuck on the last gplv2 release of binutils
> and gcc the same way I was stuck on busybox 1.2.2. ?(I'm never going to
> contribute to any GPLv3 licensed project unless paid, and I'm not
> comfortable shipping binaries of that stuff.) ?Unfortunately that means
> I can't do armv7 and the microblaze targets.

Really? Why is it so bad? Does using a GPLv3 compiler mean the
produced binaries are also under GPLv3?!

> I'm subscribed to the pcc
> mailing list and I check in on llvm/clang from time to time, and I
> really hope one of those unseats gcc. ?(That's why I was working on
> tinycc, but Grischka's permanently rendered that project irrelevant.)

It seems pcc doesn't support any ARM targets on linux and clang only
support ARMv6 and ARMv7. (I'm mostly into embedded linux because of
ARM...). I wonder when they get better ARM support.

> Using crosstool-ng would be a great workaround for that. ?They do
> toolchains so I don't have to. ?And if ALL they do is toolchains: great,
> leverage it. ?(But I need cross and native, and I'm picky about 'em, and
> I still need to build my own C library because uClibc ain't fully cooked
> yet, and ccwrap.c is a better approach than trying to get gcc's path
> logic to _EVER_ work...)

Yes, I think crosstool-NG has a clearly defined scope. BTW,
crosstool-NG also do a bit of canadian-cross:
http://ymorin.is-a-geek.org/projects/crosstool#canadian_build
http://ymorin.is-a-geek.org/hg/crosstool-ng/ct-ng.canadian/

>> Thanks for writing Aboriginal Linux, I'm very excited about it :-)
>
> Woot. ?Thanks. ?Let me know what I can do to help. :)

Thanks. I hope to give you more questions later.

Best regards,
Bj?rn Forsman

 1302191969.0

From alanpost at sunflowerriver.org  Thu Apr  7 16:49:50 2011
From: alanpost at sunflowerriver.org (Alan Post)
Date: Thu, 7 Apr 2011 10:49:50 -0600
Subject: [Aboriginal] Fwd: Re: Aboriginal Linux
In-Reply-To: <20110407211823.714eda02@U64-desktop>
References: <4D9B0256.2000502@parallels.com>
	<20110407201730.06d0f6ab@U64-desktop>
	<4D9D9805.7000001@parallels.com>
	<20110407211823.714eda02@U64-desktop>
Message-ID: <20110407164950.GB83468@alice.local>

On Thu, Apr 07, 2011 at 09:18:23PM +1000, David Seikel wrote:
> On Thu, 7 Apr 2011 05:55:01 -0500 Rob Landley <rlandley at parallels.com>
> >   http://landley.livejournal.com/48903.html
> 
> 500 Internal Server Error
> 

LJ is under DDoS right now, which is probably what you experienced
here:

  http://status.livejournal.com/

-Alan
-- 
.i ma'a lo bradi ku penmi gi'e du

 1302194990.0

From alessioigorbogani at gmail.com  Thu Apr  7 20:03:43 2011
From: alessioigorbogani at gmail.com (Alessio Igor Bogani)
Date: Thu, 7 Apr 2011 22:03:43 +0200
Subject: [Aboriginal] My dreams about Aboriginal
Message-ID: <BANLkTi=R3=kvmMao_Vkt6rV3mbsJ8aZE5g@mail.gmail.com>

Hi All,

What follows are my dreams about the future of Aboriginal Linux:

1) Switch to klibc (already hosted on kernel.org)
2) Migrate toybox to kernel.org and use it for replace busybox
3) Integrate linux-tiny patchset to provide the best and light Linux
embedded system (easy to study, fast to build)
4) Integrate KVM and use it when possible instead of QEMU (KVM or
Native KVM: http://thread.gmane.org/gmane.linux.kernel/1121307 not
matter)

Could these dreams become reality?

Ciao,
Alessio

P.s.
About my last email:
I'm not interested in Ubuntu bootstrap at all. In fact I'm not
interested in any big distro bootstrap. What I meant:

sudo apt-get install aboriginal-linux-{arm,mips,powerpc,x86}

 1302206623.0

From onefang at gmail.com  Thu Apr  7 21:21:06 2011
From: onefang at gmail.com (David Seikel)
Date: Fri, 8 Apr 2011 07:21:06 +1000
Subject: [Aboriginal] Couple of bugs.
Message-ID: <20110408072106.41eadb45@U64-desktop>

When using SYSIMAGE_TYPE=ext2 (or anything that is not squashfs), then
the squashfs stuff is not built.  The bootstrap stuff still tries to
use the non existent mksquashfs though.

The bootstrap tarballs are being named foo.tar.gz-i486.

Both easy enough to fix, and you might have done so already anyway.
This is with the last release.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110408/4dd58d68/attachment.sig>

From onefang at gmail.com  Thu Apr  7 21:36:06 2011
From: onefang at gmail.com (David Seikel)
Date: Fri, 8 Apr 2011 07:36:06 +1000
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <4D96BEF5.5040401@parallels.com>
References: <20110331180847.1e70e680@U64-desktop>
	<4D94771C.7070708@landley.net>
	<20110402000443.2caa5c65@U64-desktop>
	<4D96BEF5.5040401@parallels.com>
Message-ID: <20110408073606.099e6a7e@U64-desktop>

On Sat, 2 Apr 2011 01:15:17 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> On 04/01/2011 09:04 AM, David Seikel wrote:
> > I'm not volunteering, I got enough work to do.  Only dabbling in the
> > stuff I need to, and contributing back what I can.
> 
> This is why I want to do distro bootstrapping.  Maintaining package
> repositories is something _other_ people do, and do it well, and I
> want to leverage that work, not duplicate it.
> 
> Even LFS (and BLFS) are pretty much "repositories somebody else
> maintains".  They're as simple as you can get doing that (fixed
> package selection), but I don't have to work out what depends on what
> and how to configure invoke the builds.  (I just have to debug it. :)

Want me to send you the few BLFS scripts I made for my project?

> In theory Gentoo's emphasis on building from source made it a good
> candidate for bootstrapping systems on arbitrary targets.  In practice
> the way they set up their repository individual annotates every single
> package with every single platform it's been tested to run on.  (To
> add a new platform, you have to touch every package ebuild file in
> the whole tree.)
> 
> That's insanely stupid for native builds on arbitrary targets you've
> already bootstrapped on.  In order to go forward, I need to REDESIGN
> THE PORTAGE TREE and push the changes upstream.

Ouch.

> >> Readline there's a bsd-licensed replacement I'd prefer to use out
> >> of sheer spite for the FSF, and idealy busybox should export that
> >> functionality anyway.
> > 
> > I'd like to look at that readline replacement, though I'd be just as
> > happy to not need it for my current project.
> 
> There's a couple of them, but this one looks reasonable at first
> glance:
> 
>   https://github.com/antirez/linenoise

That looked good, especially the fork that made it more like a drop in
replacement for readline.  Even the authors of readline say in the man
page "It's too big and too slow."

> >>> First problem, iana-etc needs the real gawk, it uses fancy options
> >>> that the busybox awk does not support.  So put gawk back in.
> >>
> >> Poke the busybox gawk maintainer about that.
> > 
> > I may just cut iana-etc down to the few ports I actually need for
> > this project.  The next project will need it for sure.  I've not
> > actually looked at what iana-etc uses gawk for, but it does smell
> > of overkill to just create two text files.  Why not just ship the
> > two text files?
> 
> I know.  (Haven't looked at it since october, I think there was some
> kind of hammer I could hit it with to make it stop...)

It's on my hit list for this project.

> > True.  On the other hand, one of the goals of this project of mine
> > is to provide a government testing lab with something for them to
> > audit, including the development system that we use to build it
> > all, so that the lab can build it to.  The less they have to deal
> > with, the better we will be.  So at a later stage I'll very
> > carefully pull out as much stuff as I can to pare it down to the
> > bare minimum.  Simple minimisations are good enough for this early
> > in the project.  Get it to work first, then optimise away more crap
> > later.
> 
> I hope the orthogonal layers of the build system are reasonable to
> audit separately. :)

The theory I'm going with is - it's mostly well known open source
packages, audit labs probably only really have to look at my patches
and my code.


> >> yacc/bison is something mathematicians like and which I personally
> >> find conceptually unpleasant.
> > 
> > There might be something using bison that is important to me later.
> > I'll know for sure later.
> 
> You can always add more packages later.

Bison and flex not needed after all.  B-)

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110408/e2a10263/attachment.sig>

From onefang at gmail.com  Thu Apr  7 21:43:53 2011
From: onefang at gmail.com (David Seikel)
Date: Fri, 8 Apr 2011 07:43:53 +1000
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <4D9B01DC.7050102@parallels.com>
References: <20110331180847.1e70e680@U64-desktop>
	<4D94771C.7070708@landley.net>
	<20110402000443.2caa5c65@U64-desktop>
	<4D96BEF5.5040401@parallels.com>
	<20110405202405.4d285901@U64-desktop>
	<4D9B01DC.7050102@parallels.com>
Message-ID: <20110408074353.47a7221b@U64-desktop>

On Tue, 5 Apr 2011 06:49:48 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> On 04/05/2011 05:24 AM, David Seikel wrote:
> > The rest of that I'll reply to later, just want to get something
> > off my chest before sleeping.
> > 
> > On Sat, 2 Apr 2011 01:15:17 -0500 Rob Landley
> > <rlandley at parallels.com> wrote:
> > 
> >>>> Readline there's a bsd-licensed replacement I'd prefer to use out
> >>>> of sheer spite for the FSF, and idealy busybox should export that
> >>>> functionality anyway.  
> >>>
> >>> I'd like to look at that readline replacement, though I'd be just
> >>> as happy to not need it for my current project.  
> >>
> >> There's a couple of them, but this one looks reasonable at first
> >> glance:
> >>
> >>   https://github.com/antirez/linenoise
> > 
> > Turns out that lua needs readline.  Lua also needs ncurses.
> 
> I thought it was readline that needed ncurses, but I haven't really
> poked at lua building recently.  (It's on my todo list...)

Yes. readline needs ncurses, but lua needs to link to both libraries to
satisfy the readline requirement.  Looks like they have a switch (sort
of) to turn that off, for those OSes that don't have readline.  Will
be trivial to patch out.

> > BTW, is anyone else on this list, or did you loose them all and I'm
> > the first to find it?  Seems awfully quiet in here.
> 
> The old list went away when impactlinux.com went down, and I lost the
> subscription list when that happened.  I didn't get around to setting
> up a new one again until a month or so back, and I haven't cut a
> release since then so people who only pop up for the releases haven't
> particularly noticed the new list's existence yet. :)

Looks like my comment stirred up some activity.  B-)

> Speaking of which, I got permission to cc one here a couple days ago
> but I don't see it... and it's not in the web archive either.  Great,
> gmail's insane spam filtering strikes again.  Second attempt,
> modifying the message enough that hopefully gmail won't decide it's a
> duplicate...

Actually, both made it through.  Just the gmail auto archive that my
read only / write only hack works around must have ate them while they
tried to make it back to you.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110408/82e77d08/attachment.sig>

From rlandley at parallels.com  Thu Apr  7 23:40:56 2011
From: rlandley at parallels.com (Rob Landley)
Date: Thu, 7 Apr 2011 18:40:56 -0500
Subject: [Aboriginal] undefined reference to `finite'
In-Reply-To: <20110408011719.52c516d1@U64-desktop>
References: <20110405201205.5542b8c9@U64-desktop>	<4D9AFE66.7040906@parallels.com>	<20110407232134.09936acb@U64-desktop>
	<20110408011719.52c516d1@U64-desktop>
Message-ID: <4D9E4B88.2090400@parallels.com>

On 04/07/2011 10:17 AM, David Seikel wrote:
> On Thu, 7 Apr 2011 23:21:34 +1000 David Seikel <onefang at gmail.com>
> wrote:
>
> That did the trick.  All the libraries and development tools I think
> I'll need are now compiled.  YAY!!
> 
> Got it all in a modified version of lfs-bootstrap, so that's the entire
> thing automated.  The actual application itself can be handled by just
> one more small build script added to the end of that bootstrap.  It was
> all very easy thanks to Aboriginal Linux.  I'm ahead of schedule.

Glad it worked for you. :)

>>> By the way, have you read this yet?
>>>
>>>   http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html
>>>
>>> One of the best quick and dirty "get familiar with elf file format"
>>> things I've seen...
>>
>> Nope, but I'll look at it later.
> 
> Very interesting read.  Would have been magical if he had been able to
> squeeze a little more and get the total executable size down to 42
> bytes.

Alas, he hit hard limits.  I should retry this on current 2.6 and see if
the elf interpreter these days works the same way.

Also, last year I was poking at the kernel's elf interpreter and the
uClibc dynamic linker:

  http://landley.net/notes-2010.html#15-07-2010

I should really go back and write some proper documentation for all
that, but as with so many things my todo list sort of composts as new
layers get thrown on top of the previous priorities faster than I
actually do them.  Some stuff bubbles back up to the surface and others
sink for years at a time...

(Currently the x86_64 target isn't working with uClibc-1.0-rc3 because
it claims some code it's trying to suck into libc.so.0 didn't get built
with -fpic.  Yet more todo items to track down...)

Rob

 1302219656.0

From rlandley at parallels.com  Thu Apr  7 23:57:41 2011
From: rlandley at parallels.com (Rob Landley)
Date: Thu, 7 Apr 2011 18:57:41 -0500
Subject: [Aboriginal] My dreams about Aboriginal
In-Reply-To: <BANLkTi=R3=kvmMao_Vkt6rV3mbsJ8aZE5g@mail.gmail.com>
References: <BANLkTi=R3=kvmMao_Vkt6rV3mbsJ8aZE5g@mail.gmail.com>
Message-ID: <4D9E4F75.2090106@parallels.com>

On 04/07/2011 03:03 PM, Alessio Igor Bogani wrote:
> Hi All,
> 
> What follows are my dreams about the future of Aboriginal Linux:
> 
> 1) Switch to klibc (already hosted on kernel.org)

I looked at klibc, but Peter Anvin went crazy a few years ago and
inserted perl into the build of every single project he had anything to
do with, including the Linux kernel, syslinux, and klibc.  He seems to
think that packages which do NOT require perl to build are somehow
defective.

(I've patched it back out of the kernel, but haven't touched his other
stuff.)

My general approach to other C libraries is "build a chroot using them
natively".  I could try working up a bootstrap-klibc.hdc if you'd like.

> 2) Migrate toybox to kernel.org and use it for replace busybox

I spent years working on that, but haven't had time for it in a while.
You're welcome to grab it and run with it if you'd like.  Personally I
much prefer the design of toybox to busybox, but the practical effect is
I can't stand to work on busybox anymore.  (I tried pusing my "all
config in one file" approach to Denys, and you saw what he did with
that.  Nice guy, good engineer, I disagree with his aesthetic
sensibilities.)

> 3) Integrate linux-tiny patchset to provide the best and light Linux
> embedded system (easy to study, fast to build)

Um, which linux-tiny patchset?  Matt Mackall stopped doing it a while
back, and I thought the CELF guys were feeding theirs upstream?

For years I've wanted to create a "hello world" kernel, chopping down
Linux to just the boot wrapper and startup code necessary to switch to
C, and then just enough of the early_printk() stuff to print "hello
world" and then halt.

I ranted about that here once upon a time:

  http://www.mentby.com/rob-landley/what-happened-to-linux-tiny.html

Yet another thing I haven't had time for.  (I should really be working
on rpc.lockd and rpc.statd containerization right now.)

> 4) Integrate KVM and use it when possible instead of QEMU (KVM or
> Native KVM: http://thread.gmane.org/gmane.linux.kernel/1121307 not
> matter)

I've used kvm on x86-64, it worked fine.  All you have to do is tweak
run-emulator.sh to replace "qemu-system-x86_64" with "kvm".  (The way
that's generated is the system-image.sh script calls emulator_command()
out of sources/targets/$ARCH/settings so you could alternately edit
sources/targets/x86_64/settings to replace qemu-system-x86_64 with kvm.)

In theory _qemu_ should be doing this automatically.  In practice, the
qemu and kvm trees were only merged fairly recently and most distros
still treat them as separate packages.

> Could these dreams become reality?

Sure.  If nothing else, I try not to get in people's _way_ when they
want to do stuff.  (My goals are fairly generic, and I know where my
project _stops_.  I provide infrastructure for people to build on.  I
try to make it as easy as possible for them to do so.)

> Ciao,
> Alessio
> 
> P.s.
> About my last email:
> I'm not interested in Ubuntu bootstrap at all. In fact I'm not
> interested in any big distro bootstrap. What I meant:
> 
> sudo apt-get install aboriginal-linux-{arm,mips,powerpc,x86}

I ship prebuilt binary tarballs under GPLv2.  You can download and
extract 'em into your home directory.  Nothing stops people from making
.deb or .rpm files out of 'em, but I'm unlikely to do it because that
involves judgement calls.  (Such as "where should they be installed".)

I tried to explain how to set them up in the FAQ:

  http://landley.net/aboriginal/FAQ.html#where_start

But there's a tension between "here's the minimum you need to know" and
"here's a thousand interesting tangents you could follow" that makes
documenting this stuff really hard.  (People need a linear narrative to
learn, and this is a big tangled matrix-graph-thing.)

Basically you extract a system image tarball and a cross compiler
tarball, add the cross compiler's "bin" directory to your $PATH, cd into
the system image and run dev-environment.sh.  Then play.

Building from source is for us crazy people who want to try out new
package versions and such, which will break and need to be debugged.

Rob

 1302220661.0

From rlandley at parallels.com  Fri Apr  8 00:11:47 2011
From: rlandley at parallels.com (Rob Landley)
Date: Thu, 7 Apr 2011 19:11:47 -0500
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <20110408072106.41eadb45@U64-desktop>
References: <20110408072106.41eadb45@U64-desktop>
Message-ID: <4D9E52C3.2090908@parallels.com>

On 04/07/2011 04:21 PM, David Seikel wrote:
> When using SYSIMAGE_TYPE=ext2 (or anything that is not squashfs), then
> the squashfs stuff is not built.  The bootstrap stuff still tries to
> use the non existent mksquashfs though.

Argh.  Yeah, that's a problem.

> The bootstrap tarballs are being named foo.tar.gz-i486.

Um, details?  (I haven't seen that one.)

> Both easy enough to fix, and you might have done so already anyway.
> This is with the last release.

The first one's a design issue, which makes it tricky.

The point of host-tools.sh is to sanitize the host environment for the
rest of the build.  It's not _just_ to make sure you've got everything
you need and known working versions of each, although with many versions
of many distros out there, it's an issue.  And the bugs that produces
are often really hard to diagnose, as this recent commit attests:

  changeset:   1335:ffb07a48a960
  user:        Rob Landley <rob at landley.net>
  date:        Fri Mar 18 05:58:35 2011 -0500
  files:       sources/download_functions.sh
  description:
  If sha1sum isn't installed, download falls back through all the
  mirrors repeatedly downloading stuff, without explaining why it
  failed.

That was a fun one.  (By its nature download.sh has to run before
host-tools.sh so it runs in the unsanitized environment, so even today
it doesn't get the benefit of a known host environment.)

But the real problem host-tools addresses is _removing_ stuff from the
$PATH, making sure the various ./configure stages don't find things like
python and ncurses that your host has but the target doesn't.  Some
people think the more your development environment has installed on it
the better, but this is very much _not_ true for cross compiling.  (And
to be honest, selinux exists to make the fortune 500 happy, not to
accomplish anything useful.)

Anyway, getting back to the point at hand: host-tools.sh sets up the
environment for the rest of the build, culminating with the system
images.  This is an artifact of that: it's not setting up anything the
steps leading up to system-image.sh don't need.

The control images aren't really part of the aboriginal linux build,
they're a seperate layer that really should be their own project.
(Nothing says you can't boot up arch linux on an arm board and run the
lfs-bootstrap stuff on that.  I haven't TESTED it, of course, but
there's no real reason it shouldn't work.)

Hmmm...  host-tools is already building distcc, which also isn't really
part of the build.  I suppose squashfs can become unconditional...  but
it's ugly.  Lemme think if there's a better way to do this.  (I'd
already have split control image building out if it didn't need to use
sources/download_functions.sh...)

Rob

 1302221507.0

From rlandley at parallels.com  Fri Apr  8 00:21:05 2011
From: rlandley at parallels.com (Rob Landley)
Date: Thu, 7 Apr 2011 19:21:05 -0500
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <20110408073606.099e6a7e@U64-desktop>
References: <20110331180847.1e70e680@U64-desktop>	<4D94771C.7070708@landley.net>	<20110402000443.2caa5c65@U64-desktop>	<4D96BEF5.5040401@parallels.com>
	<20110408073606.099e6a7e@U64-desktop>
Message-ID: <4D9E54F1.5030506@parallels.com>

On 04/07/2011 04:36 PM, David Seikel wrote:
> Want me to send you the few BLFS scripts I made for my project?

Sure. :)

>> I hope the orthogonal layers of the build system are reasonable to
>> audit separately. :)
> 
> The theory I'm going with is - it's mostly well known open source
> packages, audit labs probably only really have to look at my patches
> and my code.

What they need to do and what they feel comfortable doing may be
different things.  (As I said, security nuts love busybox.)

  http://cm.bell-labs.com/who/ken/trust.html

  http://www.dwheeler.com/trusting-trust/

(Yes, that first one is the Ken Thompson who created Unix.  Of _course_
he had a backdoor in early Unix, and the way he did it was AWESOME...)

Rob

 1302222065.0

From rob at landley.net  Fri Apr  8 00:56:54 2011
From: rob at landley.net (Rob Landley)
Date: Thu, 07 Apr 2011 19:56:54 -0500
Subject: [Aboriginal] new user on the list (with questions)
In-Reply-To: <BANLkTinguSKUaA5wGWRwhrbktK1pM0sU4A@mail.gmail.com>
References: <BANLkTi=idAQ5DGXATZtpHDTY1GURmsDmHA@mail.gmail.com>	<4D9D8054.6070306@landley.net>
	<BANLkTinguSKUaA5wGWRwhrbktK1pM0sU4A@mail.gmail.com>
Message-ID: <4D9E5D56.4060809@landley.net>

On 04/07/2011 10:59 AM, Bj?rn Forsman wrote:
> Hi Rob,
> 
> Thanks for that long (and quick!) reply! Please don't think I'm rude
> just because my reply won't be nearly as long :-)
> 
> 2011/4/7 Rob Landley <rob at landley.net>:
>> On 04/07/2011 01:33 AM, Bj?rn Forsman wrote:
>>> Hi Rob,
> [snip]
>> I've actually been meaning to take openembedded and buildroot and such
>> and treat them as small distros: try to chop off their "what you build"
>> and "how you build" bits to convince them to natively build their
>> packages on an unknown target.  I.E. treat buildroot like koji or
>> catalyst: something that populates a chroot using the host's native
>> toolchain.
> 
> You mean creating buildroot and OE control-images? (I wasn't expecting
> that after the "accidental distro" thing.)

Whether or not they meant to be distros, they are distros, with existing
userbases and development communities.  Creating control images for
their distro parts would let people use them as distros without having
to rely on their ability to bootstrap new hardware.  (I.E. separating
those roles and focusing on _just_ their existence as a distro.)

I'm trying not to let _my_ project get sucked into the trap of becoming
a distro and losing focus on terraforming new hardware platforms.  (It
still seems to confuse people that you can have a tool that
intentionally does just that and nothing else.)

>>> * Where is the hg clone URL for Aboriginal Linux?
>>
>> hg clone http://landley.net/hg/aboriginal
> 
> Ah, so it's the same as the webUI! I was so close to trying that :-)

Mercurial is very simple.  Learning git first gives you a bunch of
horrible assumptions.  (Why WOULD you have two urls for the same
repository?)

>> At the top of my todo list is finishing gentoo-boostrap because A) I've
>> already done half the work for that (and 2/3 of the research), B) it
>> builds from source already.
> 
> What's missing in gentoo-bootstrap? (I haven't tried it yet.)

Um, been a while, lemme check my blog:

  http://landley.net/notes-2010.html#08-09-2010
  http://landley.net/notes-2010.html#12-09-2010
  http://landley.net/notes-2010.html#15-09-2010
  http://landley.net/notes-2010.html#01-12-2010

That should give you a flavor of it.  I haven't honestly poked at it
since last year, still trying to get lfs-boostrap to work on all
hardware targets, which means upgrading uClibc...

> It's really nice that Gentoo has one package metadata repository for
> all ARCHs (they do right?), but having a package manager written in
> python feels bloated for embedded systems.

There's a C version, it just doesn't work as well.

And their repository format is... "interesting".  It needs a redesign
for scalability.  Unfortunately, Solar's burned out and the other
maintainer of gentoo-embedded (Mike Frysinger) has such horrible
engineering taste that you can almost reliably determine the correct
course of action by what he argues AGAINST doing.

> Archlinux is pretty easy to build from source also (see Arch Build
> System). But upstream Archlinux doesn't care about other architectures
> than x86/x86_64 though, so that's a minus.

If we come up with a native bootstrap script, then they wouldn't really
have to, would they? :)

>> After that probably Debian/Ubuntu, becuase thats what I use on my laptop
>> and Ubuntu is the most popular Linux distro with somewhere around 40%
>> market share.  (And considerably higher among developer workstations.)
> 
> Great! I'm glad you're planning more bootstraps.

Yeah, gotta make the time though.

Linux From Scratch built a bunch of packages which found a lot of bugs
(the analogy of driving a flock of sheep across a minefield comes to
mind).  Unfortunately, a lot of the bugs it found are in uClibc's
pthreads implementation, which on many of the more obscure platforms
isn't reliable and makes the build unreliable.  (Make uses threads and
even make -j 1 has a worker thread which the parent waits for, and if
the locking isn't reliable stuff happens out of order every once in a
while and miscompiles stuff or breaks the build.)

  http://landley.net/notes-2010.html#17-12-2010

Since pthreads is going away in favor of NTPL, trying to debug the old
threading implementation isn't very useful.  This means my priority is
getting uClibc-NPTL to work, then making bootstrap-LFS build reliably on
all my hardware targets, then getting back to trying to make the
half-finished gentoo bootstrap work.  (I might do BLFS in there first as
low-hanging fruit, but if I can't even build the full LFS reliably on
mips or powerpc right now....)

>> By the way, I'm currently stuck on the last gplv2 release of binutils
>> and gcc the same way I was stuck on busybox 1.2.2.  (I'm never going to
>> contribute to any GPLv3 licensed project unless paid, and I'm not
>> comfortable shipping binaries of that stuff.)  Unfortunately that means
>> I can't do armv7 and the microblaze targets.
> 
> Really? Why is it so bad? Does using a GPLv3 compiler mean the
> produced binaries are also under GPLv3?!

Well, yes:

  http://lwn.net/Articles/343608/

But for me it's more:

  http://www.landley.net/notes-2006.html#03-12-2006

>> I'm subscribed to the pcc
>> mailing list and I check in on llvm/clang from time to time, and I
>> really hope one of those unseats gcc.  (That's why I was working on
>> tinycc, but Grischka's permanently rendered that project irrelevant.)
> 
> It seems pcc doesn't support any ARM targets on linux and clang only
> support ARMv6 and ARMv7. (I'm mostly into embedded linux because of
> ARM...). I wonder when they get better ARM support.

They're new.  More target support should come in time.

Really what I wanted to do was this:

http://tree.celinuxforum.org/pipermail/celinux-dev/2009-December/001924.html

Leverage QEMU's target support in tcc, and suddenly yo have a compiler
that supports everything.  (It may not produce very efficient output,
but if the whole compiler is a single 100k binary that's forgivable.)

Yet ANOTHER project I haven't had time to pursue.  It'd be nice if
somebody would...

>> Using crosstool-ng would be a great workaround for that.  They do
>> toolchains so I don't have to.  And if ALL they do is toolchains: great,
>> leverage it.  (But I need cross and native, and I'm picky about 'em, and
>> I still need to build my own C library because uClibc ain't fully cooked
>> yet, and ccwrap.c is a better approach than trying to get gcc's path
>> logic to _EVER_ work...)
> 
> Yes, I think crosstool-NG has a clearly defined scope. BTW,
> crosstool-NG also do a bit of canadian-cross:
> http://ymorin.is-a-geek.org/projects/crosstool#canadian_build
> http://ymorin.is-a-geek.org/hg/crosstool-ng/ct-ng.canadian/

My native-compiler.sh and cross-compiler.sh scripts are technically
doing canadian cross as well.  That's a normal part of building toolchains.

In 2008 I hosted a compiler BOF at OLS that turned into me ranting for
an hour (didn't _mean_ to but they kept asking me questions as if I knew
stuff):

  http://landley.net/talks/ols2008-rob-landley-linux-compiler.ogg

One of the topics was how compilers SHOULD work.  (Read input files,
produce output files.  Just like a docbook to pdf converter.  There's
nothing special about this, and compiler developers who think there IS
are doing it wrong.  Thus half the distinction between "native" and
"cross" compiling is needless confusion based on people who think what
they're doing is magic when it isn't.  I blame the FSF, as usual.)

The hard part of design is figuring out what NOT to do, and how.

Rob

 1302224214.0

From rob at landley.net  Fri Apr  8 15:28:17 2011
From: rob at landley.net (Rob Landley)
Date: Fri, 08 Apr 2011 10:28:17 -0500
Subject: [Aboriginal] My dreams about Aboriginal
In-Reply-To: <BANLkTimzQ2yP9ntApsdNZj+zYSSPw+offw@mail.gmail.com>
References: <BANLkTi=R3=kvmMao_Vkt6rV3mbsJ8aZE5g@mail.gmail.com>	<4D9E4F75.2090106@parallels.com>
	<BANLkTimzQ2yP9ntApsdNZj+zYSSPw+offw@mail.gmail.com>
Message-ID: <4D9F2991.7070909@landley.net>

On 04/08/2011 03:52 AM, Alessio Igor Bogani wrote:
> Rob,
> 
> 2011/4/8 Rob Landley <rlandley at parallels.com>:
> [...]
>>> 1) Switch to klibc (already hosted on kernel.org)
>>
>> I looked at klibc, but Peter Anvin went crazy a few years ago and
>> inserted perl into the build of every single project he had anything to
> [...]
> 
> Is Perl the only obstacle?

Perl is the big one, but the fact it's not a real C library is another.

It's not intended to be a fully functional libc capable of having things
like busybox built against it, it's basically a stub C library with just
enough functionality to build a couple small initramfs utilities and
then switch-root to a real filesystem.  (It wraps the system calls and
provides printf.)

Maybe its' design goals migrated, but last time I tested it you couldn't
build anything real against it.

>>> 2) Migrate toybox to kernel.org and use it for replace busybox
>>
>> I spent years working on that, but haven't had time for it in a while.
>> You're welcome to grab it and run with it if you'd like.  Personally I
>> much prefer the design of toybox to busybox, but the practical effect is
>> I can't stand to work on busybox anymore.
> 
> Did you mean "I can't stand to work on toybox anymore", didn't you?

No, I mean I can't stand to work on busybox anymore.

Working on toybox is "fun but pointless" since I'm up against a team of
a dozen programmers with a 10 year headstart.  I like the toybox
infrastructure, but I'm reinventing the wheel.  This has already been
done, and the existing one is "good enough" for the vast majority of the
audience.

Working on busybox is a slog through ugly code.  Back under Erik it was
reasonably tidy, and I spent a lot of time just cleaning stuff up.
Denys cares more about getting stuff to work today than he does about
making it simple or easy to read.  It's a nest of tangled #ifdefs, bits
of it have sprouted windows support, the main() function for the
multiplexer is no longer at the top level but buried down in libbb
somewhere for no apparent reason, it's got abominations like the
applet_main() prototype right before the applet definition (why the HELL
would anyone think repeating the same information twice in a row would
be an improvement, it annotates things with magic macros like
MAIN_EXTERNALLY_VISIBLE and UNUSED_PARAM that are simultaneously subtle
and pointless (I.E. most developers don't understand what they do and
what would happen if they weren't there, and what they do isn't worth
doing anyway), it not only still has multiple shell implementations but
it's developed forked implementations of all sorts of OTHER applets
(editors/patch.c, editors/patch_bbox.c, editors/patch_toybox.c,
networking/nc.c, networking/nc_bloaty.c, coreutils/od.c,
coreutils/od_bloaty.c...)

I know Denys is attempting to satisfy the needs of a large diverse
userbase, but Alan Cox once told me "a maintainer's job is to say no",
and he's not doing nearly enough of that.  To me the point of busybox
was always that it was simple and clean.  Being small was a _part_ of
being simple and clean.  Being simple made it small, and it was small
because it was simple.  (Being fast or full-featured wasn't THERE when I
started.  When I first picked busybox up it didn't WORK for most of what
I tried, and the performance of what it did have utterly sucked, yet I
was attracted to it anyway.)  Now it has to be all things to all people,
and I am apparently not one of those people anymore.

I use it because it's way better than the gnu bloatware, and because I
spent many years MAKING IT WORK and wanted to follow through on a dev
environment built around it.  (Which was a crazy goal when I started,
and required me becoming project maintainer for a while to make happen,
but now works fine.  Modulo occasionally having to give 'em a kick, ala
aboriginal hg 1318, 1180, 996...)

Note that I have infrastructure in aboriginal linux
(more/record-commands.sh) which figures out the list of commands that
are actually _used_ by the build.  I've even documented it, see:

  http://landley.net/aboriginal/FAQ.html#debug_logging

And look for "using the command logging wrapper".

And yes, this can be used natively in the resulting filesystem to
determine which commands a given package build uses, although I haven't
automated installing it yet.  Possibly I should.

I was using this during toybox development to give me a todo list.

  http://landley.net/notes-2008.html#23-01-2008

And later:

  http://landley.net/notes-2008.html#04-07-2008

>>> 3) Integrate linux-tiny patchset to provide the best and light Linux
>>> embedded system (easy to study, fast to build)
>>
>> Um, which linux-tiny patchset?  Matt Mackall stopped doing it a while
>> back, and I thought the CELF guys were feeding theirs upstream?
> 
> AFAIK The CELF (or should I say LF?) would want to revamp effort on that.

CELF was great when it was Tim Bird backed by Sony.  (Everything
interesting actually gets done by somebody _specific_.  Organizations
are abstractions, they can't implement.)

Now it's a faceless corporate entity that was founded to supply Linus
Torvalds a steady vendor-neutral paycheck but wasn't satisfied to do
JUST that, and suffered years of feature creep until it merged with a
standards body in an explosion of bureaucracy.

The Linux Foundation is to Linux what AOL was to the Internet in the
1990's.  The truly clueless aren't comfortable unless they think
somebody owns it and is responsible for it, and they want to pay that
entity money to give them the illusion of control.

>>> I'm not interested in Ubuntu bootstrap at all. In fact I'm not
>>> interested in any big distro bootstrap. What I meant:
>>>
>>> sudo apt-get install aboriginal-linux-{arm,mips,powerpc,x86}
>>
>> I ship prebuilt binary tarballs under GPLv2.  You can download and
> [...]
>> Basically you extract a system image tarball and a cross compiler
>> tarball, add the cross compiler's "bin" directory to your $PATH, cd into
>> the system image and run dev-environment.sh.  Then play.
> 
> The important point here is:
> Do you will ever provide binary tarball for every combinations of
> arch, compiler, bootstrap and so on?

I tried to document this in:

  http://landley.net/aboriginal/downloads/binaries/README

I provide one instance of each type of tarball per $ARCH.  I'm only
using one compiler (binutils, gcc, kernel headers, uclibc), although it
comes in cross and native versions.  (Cross is from i686 which runs on
x86_64.  You can build other types using the build scripts, although I
doubt the gcc guys have really tested cross compiling sparc from an arm
host.)

I don't do bootloaders, installing it on real hardware is Not My
Problem.  (Outside the scope of the project.)

I have the hw- overlay mechanism if you want to create your own
derivative targets (see sources/targets/hw-tct-hammer for an example,
it's mostly good for creating a second system image from the base root
filesystem using a different kernel).

Also, you can replace my bootscript using SIMPLE_ROOT_OVERLAY (it's a
configuration environment variable, which means the "config" file at the
top level of the build scripts documents its usage.)

> My major concerns are about Aboriginal project is the total absence of
> a clear vision:

Sigh.  I have a clear vision (since 2008-ish, anyway), and I've tried to
articulate it.  Apparently I haven't managed.  One such attempt was:

  http://landley.net/aboriginal/about.html

> What is Aboriginal made for? What is goot at? What are
> the key features? What is the future direction? What are the best
> things that project care to provide (speed vs size, simplicity vs
> flexibility)? What are the differences with Yocto, OE, buildroot,
> crosstool, PTXdist, ELDK, Scratchbox, emdebian and so on?

The point of Aboriginal Linux is to get a minimal native Linux
development environment up and running on a new platform, and then get
out of your way.  The motto "we cross compile so you don't have to"
means once you have that native development environment you should be
able to build anything else you need natively on your target hardware
(or under an emulator for that target).

You can still use a cross compiler via distcc to speed up native builds
by leveraging fast cheap x86 hardware, but this is purely optional and
your package builds still act like fully native package builds when you
do this; you never have to mess with cross compiler prefixes or the many
flavors of host/target confusion just because you're using distcc.

The simplest Linux system I've ever managed to come up with that was
capable of building itself under itself is seven packages (linux,
uClibc, binutils, gcc, busybox, bash, and make), and we cross compile
those seven packages, then package them together with a kernel and a
boot script into a bootable system image configured for use with QEMU.

And then we STOP.  We are not a distro, we don't care what final
packages you build to actually deploy on your target.  We just give you
a build environment, what you build with it is none of our business.
(Making sure you CAN build any package you try to is important,
otherwise we have a bug we need to fix.  But package selection and
package management and such?  Beyond the scope of hte project.  We
provide build automation infrastructure in the control-image.hdc stuff,
and we're working to provide example images that bootstrap popular
distros.  But those are just examples, and that collection of examples
may spin off into a separate project.)

> I'm not asking you to reply on all these answers. I would want know
> only a _clear_ vision about Aboriginal project so I can decide _alone_
> if Aboriginal is good for me.
> 
> I really damned like how you have made Aboriginal but, frankly, a
> project like this without a clear vision drive me to looking for
> something else.

As far as I'm concerned, a design has to know what it DOESN'T do as much
as what it does.  "Where are the edges of this project?"

This project is divided into a series of orthogonal layers, each of
which tries hard not to leak into or depend unnecessarily on the others.
 (As described in presentation.pdf in the downloads directory.)  These
layers are implemented in separate scripts, which build.sh calls in
order, and are described here:

  http://landley.net/aboriginal/downloads/README

(Notice: the README in downloads describes the build scripts, I.E. the
source code.  The README in downloads/binaries describes the binary
tarballs, I.E. the output of the build.)

Here are the stages bulid.sh calls:

  download.sh - download the source packages for the rest of the build

  host-tools.sh - build host binaries used by the rest of the build

  simple-cross-compiler.sh - build a "stage 1" compiler

  cross-compiler.sh - build a "stage 2" compiler

  native-compiler.sh - build a ("stage 2") native compiler

  simple-root-filesystem.sh - just enough to boot to a shell prompt

  root-filesystem.sh - combine native-compiler + simple-root-filesystem

  system-image.sh - package root filesystem into something qemu can boot
                    (also builds kernel configured for use with qemu).

Those layers are orthogonal and most of them are optional.  If you
already have a "packages" directory with source tarballs in it, all
download.sh does is confirm the checksums.  You don't have to use THIS
way to download the packages.

You can skip host-tools.sh if you've set your host up sufficiently
carefully.  You don't have to use THIS way to set up the host environment.

If you already have a cross compiler (ala crosstool-ng) you can use that
instead of building one with this script.  You don't have to use THIS
cross compiler.

You never need cross-compiler.sh (it's there to create a portable one I
can upload to the website, the rest of the build doesn't use it and by
default build.sh won't call it unless you set a config variable).

If you just want a shell prompt with no dev environment skip
native-compiler.sh (setting NO_NATIVE_COMPILER tells build.sh to do
this).  Or if you have a different native compiler (perhaps from
crosstool-ng) you can use that one instead of building one with this script.

If you just want to build toolchains and not a root filesystem skip
simple-root-filesystem.sh.  Or if you have a different root filesystem
you want to use, put it in "build/simple-root-filesystem-$ARCH" and the
build will try to use it.

The root-filesystem.sh script just combines the previous two output
directories, it's just two instances of "cp" with an rm -rf and a call
to strip.

And if you want to chroot into your root filesystem or install it on
target yourself, system-image.sh is irrelevant.  Or you can use it to
package up some other root filesystem (just copy it to
"build/root-filesystem-$ARCH", this script doesn't particularly care
what's in there) and stick a kernel on it.

That's the path from point A to point B.  "I have a small collection of
bash scripts" to "I have a system image qemu can boot".  That's pretty
much what Aboriginal Linux was designed to do.  That's the edge of the
project, at which point we start answering a lot of questions with "not
our problem".

Now beyond that, we do have the control images.  The script

  more/build-control-images.sh

Builds all of those.  Basically when a system image boots up, its init
script checks to see if /mnt/init exists, and if so it transfers control
to that at the end instead of running a shell prompt.  (Actualy it waits
3 seconds for you to press a key first, in which case you get your shell
prompt anyway but now with the control image mounted.)

In each system image there are three shell script that can boot qemu in
different ways:

  run-emulator.sh - root filesystem on /dev/hda, mounted on /
  dev-environment.sh - 2 gig ext2 image on /dev/hdb, mounted on /home
  native-build.sh - control image on /dev/hdc, mounted on /mnt

Note that native-build.sh is a wrapper around dev-environment.sh, which
is a wrapper around run-emulator.sh.  Each one adds some configuration
then calls the next simpler one.

This is generic build automation infrastructure: aboriginal linux
provides a way to run builds without human intervention (from a cron job
or some such), but doesn't care what you're building.  You can supply
any control image you like, it gets mounted on /dev/hdc and its "init"
program gets run.  It can contain anything, and do anything, you like.

On the one hand, maintaining lots of example control images is none of
Aboriginal Linux's business.  On the other hand, building things like
Linux From Scratch under the resulting system is squeezing a lot of bugs
out of the build system (especially uClibc).  So for the moment it's
part of the smae project, it may be broken into a separate project layer
on.  It IS definitely an orthogonal layer you can ignore, though.

So providing a way to run control images: part of aboriginal linux.
Providing specific control images: not really part of aboriginal linux
(in the long term, anyway).  Worth doing in some context, being used as
debugging test cases right now, and a certain number of EXAMPLE control
images are excusable to show you how to make 'em.

But once we have a critical mass of control images, I'll probably split
that part out into a separate project and move them out of Aboriginal
Linux.  I try to make them not depend on aboriginal linux (you should be
able to run 'em under any arbitrary distro's dev environment, not that
I've tested this), and I definitely don't make aboriginal linux ever
depend on anything in a control image.

> Ciao,
> Alessio

Did that help explain?

Rob

 1302276497.0

From miltonm at bga.com  Sat Apr  9 05:42:33 2011
From: miltonm at bga.com (Milton Miller)
Date: Sat, 09 Apr 2011 00:42:33 -0500
Subject: [Aboriginal] new user on the list (with questions)
In-Reply-To: <BANLkTi=idAQ5DGXATZtpHDTY1GURmsDmHA@mail.gmail.com>
Message-ID: <conv-to-git@mdm.bga.com>

[resend with subject]

On Wed Apr 6 about 23:33:57 PDT 2011, Bj??rn Forsman wrote:
> * Where is the hg clone URL for Aboriginal Linux?
> BTW, If you're considering setting up a git repo (I saw an email the
> other day), maybe this helps: http://hg-git.github.com/

Actullay I have some hg repositories at work and the one thing I find
lacking is hg view doesn't let me filter on a set of paths like gitk.
Anyways, I found this suited my needs better:

	git://repo.or.cz/fast-export.git

Its written in python and imports the Mercurial objects to access hg
data then feeds it though a pipe to git to write its .pack file; it
doesn't require third party libraries to be compiled and installed
or try to write git files directly.

Exporting the entire Aboriginal tree with git init; hg-fast-export.sh
took 7 seconds on my laptop and a git repack -a -f took less than 2 and
reduced the object store from 5k blocks to about 2K.  (I let the .git
sit next to .hg and therefore don't need any args, just .git in .hginore).

milton

 1302327753.0

From rob at landley.net  Sat Apr  9 05:55:02 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 09 Apr 2011 00:55:02 -0500
Subject: [Aboriginal] (No subject header)
In-Reply-To: <conv-to-git@mdm.bga.com>
References: <conv-to-git@mdm.bga.com>
Message-ID: <4D9FF4B6.1030009@landley.net>

On 04/09/2011 12:39 AM, Milton Miller wrote:
> On Wed Apr 6 about 23:33:57 PDT 2011, Bj??rn Forsman wrote:
>> * Where is the hg clone URL for Aboriginal Linux?
>> BTW, If you're considering setting up a git repo (I saw an email the
>> other day), maybe this helps: http://hg-git.github.com/
> 
> Actullay I have some hg repositories at work and the one thing I find
> lacking is hg view doesn't let me filter on a set of paths like gitk.
> Anyways, I found this suited my needs better:
> 
> 	git://repo.or.cz/fast-export.git

A project web page would be nice.  I have to pull up git to clone that
repo and then browse it from the command line, which is enough work to
put it on my todo list instead of me doing it right now.

> Its written in python and imports the Mercurial objects to access hg
> data then feeds it though a pipe to git to write its .pack file; it
> doesn't require third party libraries to be compiled and installed
> or try to write git files directly.
> 
> Exporting the entire Aboriginal tree with git init; hg-fast-export.sh
> took 7 seconds on my laptop and a git repack -a -f took less than 2 and
> reduced the object store from 5k blocks to about 2K.  (I let the .git
> sit next to .hg and therefore don't need any args, just .git in .hginore).
> 
> milton

Ooh, that's nice.

How is it about making on repo track another?  Can it do incremental
updates, or if I re-covert the repo will it generate the same sha1sums
each time?

Rob

 1302328502.0

From onefang at gmail.com  Sat Apr  9 09:20:28 2011
From: onefang at gmail.com (David Seikel)
Date: Sat, 9 Apr 2011 19:20:28 +1000
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <4D9E54F1.5030506@parallels.com>
References: <20110331180847.1e70e680@U64-desktop>
	<4D94771C.7070708@landley.net>
	<20110402000443.2caa5c65@U64-desktop>
	<4D96BEF5.5040401@parallels.com>
	<20110408073606.099e6a7e@U64-desktop>
	<4D9E54F1.5030506@parallels.com>
Message-ID: <20110409192028.4ffbc64b@U64-desktop>

On Thu, 7 Apr 2011 19:21:05 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> On 04/07/2011 04:36 PM, David Seikel wrote:
> > Want me to send you the few BLFS scripts I made for my project?
> 
> Sure. :)

They should be attached as text/plain, let's see if that gets it past
the various spam filters.

I think the only dependency is mc depending on glib.  For the record I
compile them in this order (after the LFS stage) - glib, mc, libjpeg,
libpng, freetype.  I actually interleave parts of EFL with that build
order, but could build them all before EFL.  I was using a just in time
policy of what to compile when, as a quick and dirty dependency
documentation.  More of BLFS is supported by EFL as optional
dependencies, I just don't think I will need them.

Midnight commander I added cos I often use that as my IDE.  Works well
over ssh.  I remember a few from the old mailing list wanted that to.
Glib may or may not be needed for my final app, but is needed for mc.
EFL includes support for it, but it's optional.  I'll just see if it
makes anything easier later.

> >> I hope the orthogonal layers of the build system are reasonable to
> >> audit separately. :)
> > 
> > The theory I'm going with is - it's mostly well known open source
> > packages, audit labs probably only really have to look at my patches
> > and my code.
> 
> What they need to do and what they feel comfortable doing may be
> different things.  (As I said, security nuts love busybox.)

This is Australia, she'll be right mate.  B-)

At any rate, the smaller the system, the better the automation, and the
better documented it is, the happier we all will be.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: freetype.sh
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110409/231d7939/attachment.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: glib.sh
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110409/231d7939/attachment-0001.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: libjpeg.sh
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110409/231d7939/attachment-0002.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: libpng.sh
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110409/231d7939/attachment-0003.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: mc.sh
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110409/231d7939/attachment-0004.ksh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110409/231d7939/attachment.sig>

From onefang at gmail.com  Sat Apr  9 09:26:41 2011
From: onefang at gmail.com (David Seikel)
Date: Sat, 9 Apr 2011 19:26:41 +1000
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <4D9E54F1.5030506@parallels.com>
References: <20110331180847.1e70e680@U64-desktop>
	<4D94771C.7070708@landley.net>
	<20110402000443.2caa5c65@U64-desktop>
	<4D96BEF5.5040401@parallels.com>
	<20110408073606.099e6a7e@U64-desktop>
	<4D9E54F1.5030506@parallels.com>
Message-ID: <20110409192641.3e7d0303@U64-desktop>

On Thu, 7 Apr 2011 19:21:05 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> On 04/07/2011 04:36 PM, David Seikel wrote:
> > Want me to send you the few BLFS scripts I made for my project?
> 
> Sure. :)

Oops, forgot to mention.  The BLFS libjpeg page was last updated in
2007, the tarball links they provide no longer work, as the libjpeg
people have moved on and did not keep the old version around.  So I
had to bump it two versions.  sigh

They also used a tarball filename that needed some massaging.  So here
are the relevant bits out of the download script (munged by email
proggy, but you can sort it out) -

URL=http://ftp.gnome.org/pub/gnome/sources/glib/2.12/glib-2.12.12.tar.bz2
\ SHA1=527bced2e6d38169c08c8712d5e3b6ebb0dc46b5 \
maybe_fork download || dienow

URL=http://www.ibiblio.org/pub/Linux/utils/file/managers/mc/mc-4.6.1.tar.gz
\ SHA1=92be0802143245e0fb7e3847d70da614cad7d5fb \
maybe_fork download || dienow

URL=http://www.ijg.org/files/jpegsrc.v8c.tar.gz \
SHA1=f0a3b88ac4db19667798bee971537eeed552bce9 \
RENAME="s/jpegsrc.v8c.tar.gz/libjpeg-8.tar.gz/" \
maybe_fork download || dienow

URL=http://downloads.sourceforge.net/libpng/libpng-1.2.29.tar.bz2 \
SHA1=ddd4c92d029705d81ba97679d1e6ee84a33057db \
maybe_fork download || dienow

URL=http://downloads.sourceforge.net/freetype/freetype-2.3.7.tar.bz2 \
SHA1=f16f849d6e739ce8842008586af36371a32ac064 \
maybe_fork download || dienow

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110409/e5bb8dc8/attachment.sig>

From onefang at gmail.com  Sat Apr  9 12:12:37 2011
From: onefang at gmail.com (David Seikel)
Date: Sat, 9 Apr 2011 22:12:37 +1000
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <4D9E52C3.2090908@parallels.com>
References: <20110408072106.41eadb45@U64-desktop>
	<4D9E52C3.2090908@parallels.com>
Message-ID: <20110409221237.3ecab345@U64-desktop>

On Thu, 7 Apr 2011 19:11:47 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> On 04/07/2011 04:21 PM, David Seikel wrote:
> > When using SYSIMAGE_TYPE=ext2 (or anything that is not squashfs),
> > then the squashfs stuff is not built.  The bootstrap stuff still
> > tries to use the non existent mksquashfs though.
> 
> Argh.  Yeah, that's a problem.
> 
> > The bootstrap tarballs are being named foo.tar.gz-i486.
> 
> Um, details?  (I haven't seen that one.)

In sources/control-images/bootstrap-skeleton/mnt/functions.sh there is
this -

upload_result()
{
  ftpput $FTP_SERVER -P $FTP_PORT "$1-$HOST" "$1"
}

The "$1-$HOST" part adds the target platform (in my case i486) at the
end of the tarball name.  So it comes AFTER the .tar.gz part of the
name.  That part of the file name should be added in during the tar
command in init.sh.

> > Both easy enough to fix, and you might have done so already anyway.
> > This is with the last release.
> 
> The first one's a design issue, which makes it tricky.

<snip> 

> Hmmm...  host-tools is already building distcc, which also isn't
> really part of the build.  I suppose squashfs can become
> unconditional...  but it's ugly.  Lemme think if there's a better way
> to do this.  (I'd already have split control image building out if it
> didn't need to use sources/download_functions.sh...)

I would fix it up the other way.  This problem happens when the user is
asking for ext2 images.  So the control images should honour that and
make ext2 images.  Then it does not matter that mksquashfs is not
built, the user did not want it anyway.

Having said that, my quick and dirty hack to get back to working has to
make mksquashfs unconditionally built.  I'm leaving a proper fix up to
you.  At least until I sort out how I'm gonna make a bootable image
file I can just dd to a CF card; one that includes multiple
partitions.  I'm not liking the options for doing that so far.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110409/b7cee800/attachment.sig>

From rlandley at parallels.com  Sat Apr  9 13:58:57 2011
From: rlandley at parallels.com (Rob Landley)
Date: Sat, 9 Apr 2011 08:58:57 -0500
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <20110409192641.3e7d0303@U64-desktop>
References: <20110331180847.1e70e680@U64-desktop>	<4D94771C.7070708@landley.net>	<20110402000443.2caa5c65@U64-desktop>	<4D96BEF5.5040401@parallels.com>	<20110408073606.099e6a7e@U64-desktop>	<4D9E54F1.5030506@parallels.com>
	<20110409192641.3e7d0303@U64-desktop>
Message-ID: <4DA06621.4060304@parallels.com>

On 04/09/2011 04:26 AM, David Seikel wrote:
> On Thu, 7 Apr 2011 19:21:05 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
>> On 04/07/2011 04:36 PM, David Seikel wrote:
>>> Want me to send you the few BLFS scripts I made for my project?
>>
>> Sure. :)
> 
> Oops, forgot to mention.  The BLFS libjpeg page was last updated in
> 2007, the tarball links they provide no longer work, as the libjpeg
> people have moved on and did not keep the old version around.  So I
> had to bump it two versions.  sigh
> 
> They also used a tarball filename that needed some massaging.  So here
> are the relevant bits out of the download script (munged by email
> proggy, but you can sort it out) -

Sigh.  I need to do something about the control image generation logic.
 It should be possible to make a new one in a self-contained way, and
the common infrastructure you draw from should be cleanly factored out.

Right now, most of it is in sources/control-images but it's not in a
unified format where listing that directory makes it obvious what the
list of control images _are_, or how to add a new one.  The common
infrastructure is partly in sources/include.sh, partly in
sources/download_functions.sh, partly in
sources/control-images/bootstrap-skeleton, and then there's
more/build-control-images.sh...

I do have a sources/control-images/bootstrap-skeleton/README file, but
who would notice that and it's still too complicated.

Right, time to factor this out and clean it up, and probably create a
new repository for it...

Rob

 1302357537.0

From rlandley at parallels.com  Sat Apr  9 14:16:56 2011
From: rlandley at parallels.com (Rob Landley)
Date: Sat, 9 Apr 2011 09:16:56 -0500
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <20110409221237.3ecab345@U64-desktop>
References: <20110408072106.41eadb45@U64-desktop>	<4D9E52C3.2090908@parallels.com>
	<20110409221237.3ecab345@U64-desktop>
Message-ID: <4DA06A58.50101@parallels.com>

On 04/09/2011 07:12 AM, David Seikel wrote:
> On Thu, 7 Apr 2011 19:11:47 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
>> On 04/07/2011 04:21 PM, David Seikel wrote:
>>> When using SYSIMAGE_TYPE=ext2 (or anything that is not squashfs),
>>> then the squashfs stuff is not built.  The bootstrap stuff still
>>> tries to use the non existent mksquashfs though.
>>
>> Argh.  Yeah, that's a problem.

Which ties into the "breaking it out into a separate project" issue,
which is a design problem rather than an implementation detail.

There's shared infrastructure that's convenient to leverage, but
conceptually this isn't part of aboriginal linux's design, but having
_example_ images to demonstrate the automated native build
infrastructure (and how to take advantage of it) probably would be, but
this is a can of worms and the way to handle cans of worms is to take
them outside...

Somewhere there needs to be a website describing how to bootstrap debian
and such.  Distros that expect to build under themselves have an
undocumented set of environmental dependencies, which _can_ be manually
satisfied to let you build Fedora under Gentoo and such (if by tweaking
the heck out of a chroot).  Somewhere there should be a website
documenting those environmental dependencies, and the best kind of
documentation for this sort of thing is the executable kind that you can
regression test.

In the absence of somebody else doing that (and I've been waiting
several years now and nobody's beaten me to it yet, you'd think the
Linux From Scratch guys would be interested but not so far), the control
images are likely to grow into doing that.  And they are NOT doing so
within the confines of the aboriginal linux project, because that's not
what this does.

So I've got a bit of egg-unscrambling to do, separating stuff out and
cleaning it up.  sigh...

>>> The bootstrap tarballs are being named foo.tar.gz-i486.
>>
>> Um, details?  (I haven't seen that one.)
> 
> In sources/control-images/bootstrap-skeleton/mnt/functions.sh there is
> this -
> 
> upload_result()
> {
>   ftpput $FTP_SERVER -P $FTP_PORT "$1-$HOST" "$1"
> }
> 
> The "$1-$HOST" part adds the target platform (in my case i486) at the
> end of the tarball name.  So it comes AFTER the .tar.gz part of the
> name.  That part of the file name should be added in during the tar
> command in init.sh.

Good point.  I cut and pasted that from static-tools.sh building
dropbear and strace.  (And once upon a time busybox, which requires a
postfix on the name "busybox" rather than a prefix, because that's what
Erik Andersen wanted way back when.  Except I cross compile a static
busybox by default, so building one natively is slightly silly.)

I need to figure out what infrastructure goes into aboriginal linux and
what goes into the separate control-images.  (Automative native build
control images.  Distro bootstrapping.  I suck at naming things, anybody
have a suggestion?  They can share the mailing list until there's enough
traffic ot justify splitting it off, but I have to name the mercurial
repository in order to put it online...)

>>> Both easy enough to fix, and you might have done so already anyway.
>>> This is with the last release.
>>
>> The first one's a design issue, which makes it tricky.
> 
> <snip> 
> 
>> Hmmm...  host-tools is already building distcc, which also isn't
>> really part of the build.  I suppose squashfs can become
>> unconditional...  but it's ugly.  Lemme think if there's a better way
>> to do this.  (I'd already have split control image building out if it
>> didn't need to use sources/download_functions.sh...)
> 
> I would fix it up the other way.  This problem happens when the user is
> asking for ext2 images.  So the control images should honour that and
> make ext2 images.  Then it does not matter that mksquashfs is not
> built, the user did not want it anyway.

Which argues for breaking system-image.sh into two parts.  (It's always
been kind of ugly, but initramfs is a form of packaging and that
requires building a kernel.  Plus building a kernel, by itself, doesn't
quite justify having its own script.  Or it does, but it would be in
sources/sections, not at the top level.)

It would be nice if the control-image infrastructure could use the
multiple filesystem format awareness of system-image, but really it's a
big if/else staircase with some hardwired hacks for /dev support.  (Read
the script, I'm open to suggestions on how to improve it...)

The problem is never implementation being too hard, the problem is
always figuring out what the right thing to do is.  Implementing it's
the easy bit (if often fiddly and time consuming)...

> Having said that, my quick and dirty hack to get back to working has to
> make mksquashfs unconditionally built.  I'm leaving a proper fix up to
> you.  At least until I sort out how I'm gonna make a bootable image
> file I can just dd to a CF card; one that includes multiple
> partitions.  I'm not liking the options for doing that so far.

SYSIMAGE_TYPE=initramfs

I've pondered adding jffs2 and yaffs2 and logfs and so on, but as you
can see there isn't a clear winner here, and to format one of these
suckers you have to know the flash erase block size and such which
doesn't abstract cleanly so far when I've poked at it...

Rob

 1302358616.0

From onefang at gmail.com  Sat Apr  9 15:06:34 2011
From: onefang at gmail.com (David Seikel)
Date: Sun, 10 Apr 2011 01:06:34 +1000
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <4DA06A58.50101@parallels.com>
References: <20110408072106.41eadb45@U64-desktop>
	<4D9E52C3.2090908@parallels.com>
	<20110409221237.3ecab345@U64-desktop>
	<4DA06A58.50101@parallels.com>
Message-ID: <20110410010634.4ca8ef77@U64-desktop>

On Sat, 9 Apr 2011 09:16:56 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> I need to figure out what infrastructure goes into aboriginal linux
> and what goes into the separate control-images.  (Automative native
> build control images.  Distro bootstrapping.  I suck at naming
> things, anybody have a suggestion?  They can share the mailing list
> until there's enough traffic ot justify splitting it off, but I have
> to name the mercurial repository in order to put it online...)

Rainbow serpent?  "moves through water and rain, shaping landscapes,
naming and singing of places, swallowing and sometimes drowning people;
strengthening the knowledgeable with rainmaking and healing powers;
blighting others with sores, weakness, illness, and death."

The control images could be both examples, and bootstrap steps to
the popular distros you have already mentioned.  Stopping short of
being distros or distro builders, and letting the distro maintainers
themselves deal with the next step once we get them bootstrapped.
Though perhaps a full BLFS build is on the cards, since that has a
limit on the packages, is a good shake down, and a good match.

Could be both separate, and being able to use Aboriginal Linux as the
build environment by having the two side by side, so people can run
both with simple scripts, then use the results to build some sort of
bootable image, or keep running it under qemu.

Could also have the option of combining it into one image, and not two
where one is mounted automatically, then have to manually chroot into it
to actually do anything with the bootstrap under qemu.

I'd probably dump a EFL on frame buffer, and EFL on X minimal bootstrap
into the EFL repo, as an example to others to not start dumping random
collections of bootstraps into your repos.  They already have an
OpenEmbedded area in their SVN.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110410/577d441e/attachment.sig>

From rlandley at parallels.com  Sat Apr  9 15:35:56 2011
From: rlandley at parallels.com (Rob Landley)
Date: Sat, 9 Apr 2011 10:35:56 -0500
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <20110410010634.4ca8ef77@U64-desktop>
References: <20110408072106.41eadb45@U64-desktop>	<4D9E52C3.2090908@parallels.com>	<20110409221237.3ecab345@U64-desktop>	<4DA06A58.50101@parallels.com>
	<20110410010634.4ca8ef77@U64-desktop>
Message-ID: <4DA07CDC.50100@parallels.com>

On 04/09/2011 10:06 AM, David Seikel wrote:
> On Sat, 9 Apr 2011 09:16:56 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
>> I need to figure out what infrastructure goes into aboriginal linux
>> and what goes into the separate control-images.  (Automative native
>> build control images.  Distro bootstrapping.  I suck at naming
>> things, anybody have a suggestion?  They can share the mailing list
>> until there's enough traffic ot justify splitting it off, but I have
>> to name the mercurial repository in order to put it online...)
> 
> Rainbow serpent?  "moves through water and rain, shaping landscapes,
> naming and singing of places, swallowing and sometimes drowning people;
> strengthening the knowledgeable with rainmaking and healing powers;
> blighting others with sores, weakness, illness, and death."

Never heard of it before.

> The control images could be both examples, and bootstrap steps to
> the popular distros you have already mentioned.  Stopping short of
> being distros or distro builders, and letting the distro maintainers
> themselves deal with the next step once we get them bootstrapped.
> Though perhaps a full BLFS build is on the cards, since that has a
> limit on the packages, is a good shake down, and a good match.

Yup.  There's good reasons to _do_ it, it's just the scope of the
control images heap could grow larger than the rest of aboriginal linux
combined.  How many distros are there?

> Could be both separate, and being able to use Aboriginal Linux as the
> build environment by having the two side by side, so people can run
> both with simple scripts, then use the results to build some sort of
> bootable image, or keep running it under qemu.
> 
> Could also have the option of combining it into one image, and not two
> where one is mounted automatically, then have to manually chroot into it
> to actually do anything with the bootstrap under qemu.

The control images are target independent, the system images are target
specific.  (And the control images can be pretty darn big, the LFS one
is 350 megs.)  It makes sense to me to keep them separate.

If you use the run-emulator.sh, it runs the build automatically, but
there's a 3 second delay where it prompts you to press a key to get a
shell prompt, and only starts the build if you don't.

I can easily change how that works, but it's really a user interface
question.  (Aesthetic issue, not necessarily a one true answer there.)

> I'd probably dump a EFL on frame buffer, and EFL on X minimal bootstrap
> into the EFL repo, as an example to others to not start dumping random
> collections of bootstraps into your repos.  They already have an
> OpenEmbedded area in their SVN.

I do want to set up a vnc desktop export.  In theory x.org does this
built-in, in practice I've misplaced the instructions I got it to work
with last time.  (Maybe it was in BLFS somewhere...)

Rob

 1302363356.0

From onefang at gmail.com  Sat Apr  9 15:52:12 2011
From: onefang at gmail.com (David Seikel)
Date: Sun, 10 Apr 2011 01:52:12 +1000
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <4DA07CDC.50100@parallels.com>
References: <20110408072106.41eadb45@U64-desktop>
	<4D9E52C3.2090908@parallels.com>
	<20110409221237.3ecab345@U64-desktop>
	<4DA06A58.50101@parallels.com>
	<20110410010634.4ca8ef77@U64-desktop>
	<4DA07CDC.50100@parallels.com>
Message-ID: <20110410015212.24f9a42c@U64-desktop>

On Sat, 9 Apr 2011 10:35:56 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> On 04/09/2011 10:06 AM, David Seikel wrote:
> > On Sat, 9 Apr 2011 09:16:56 -0500 Rob Landley
> > <rlandley at parallels.com> wrote:
> > 
> >> I need to figure out what infrastructure goes into aboriginal linux
> >> and what goes into the separate control-images.  (Automative native
> >> build control images.  Distro bootstrapping.  I suck at naming
> >> things, anybody have a suggestion?  They can share the mailing list
> >> until there's enough traffic ot justify splitting it off, but I
> >> have to name the mercurial repository in order to put it online...)
> > 
> > Rainbow serpent?  "moves through water and rain, shaping landscapes,
> > naming and singing of places, swallowing and sometimes drowning
> > people; strengthening the knowledgeable with rainmaking and healing
> > powers; blighting others with sores, weakness, illness, and death."
> 
> Never heard of it before.

Australian aboriginal mythology that is reasonably common across
Australia, and quite well known here.

> > The control images could be both examples, and bootstrap steps to
> > the popular distros you have already mentioned.  Stopping short of
> > being distros or distro builders, and letting the distro maintainers
> > themselves deal with the next step once we get them bootstrapped.
> > Though perhaps a full BLFS build is on the cards, since that has a
> > limit on the packages, is a good shake down, and a good match.
> 
> Yup.  There's good reasons to _do_ it, it's just the scope of the
> control images heap could grow larger than the rest of aboriginal
> linux combined.  How many distros are there?

That's why I said "popular distros you have already mentioned".  That's
where you draw the line and say no.  LFS, Gentoo, Debian, Red Hat, and
Ubuntu.  Maybe ArchLinux as well.  Certainly not Android.

> > Could be both separate, and being able to use Aboriginal Linux as
> > the build environment by having the two side by side, so people can
> > run both with simple scripts, then use the results to build some
> > sort of bootable image, or keep running it under qemu.
> > 
> > Could also have the option of combining it into one image, and not
> > two where one is mounted automatically, then have to manually
> > chroot into it to actually do anything with the bootstrap under
> > qemu.
> 
> The control images are target independent, the system images are
> target specific.  (And the control images can be pretty darn big, the
> LFS one is 350 megs.)  It makes sense to me to keep them separate.

Eventually a lot of users of the system will want to build bootable
images, plus for the likes of LFS, include the toolchain that has been
used so far.

> If you use the run-emulator.sh, it runs the build automatically, but
> there's a 3 second delay where it prompts you to press a key to get a
> shell prompt, and only starts the build if you don't.
> 
> I can easily change how that works, but it's really a user interface
> question.  (Aesthetic issue, not necessarily a one true answer there.)

Does it do the chroot step into the bootstrap image when you press
that key?

> > I'd probably dump a EFL on frame buffer, and EFL on X minimal
> > bootstrap into the EFL repo, as an example to others to not start
> > dumping random collections of bootstraps into your repos.  They
> > already have an OpenEmbedded area in their SVN.
> 
> I do want to set up a vnc desktop export.  In theory x.org does this
> built-in, in practice I've misplaced the instructions I got it to work
> with last time.  (Maybe it was in BLFS somewhere...)

Or don't pass -nographic to qemu, which I already have a modified
run-emulator.sh doing.  Something I'll need to play with more for this
project.


Think I have come up with an acceptable way for me to create a multi
partition bootable image suitable for DDing onto a CF card, without
having to use root on the host.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110410/1bd5870b/attachment.sig>

From rlandley at parallels.com  Sun Apr 10 06:11:20 2011
From: rlandley at parallels.com (Rob Landley)
Date: Sun, 10 Apr 2011 01:11:20 -0500
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <20110410015212.24f9a42c@U64-desktop>
References: <20110408072106.41eadb45@U64-desktop>	<4D9E52C3.2090908@parallels.com>	<20110409221237.3ecab345@U64-desktop>	<4DA06A58.50101@parallels.com>	<20110410010634.4ca8ef77@U64-desktop>	<4DA07CDC.50100@parallels.com>
	<20110410015212.24f9a42c@U64-desktop>
Message-ID: <4DA14A08.7040002@parallels.com>

On 04/09/2011 10:52 AM, David Seikel wrote:
> On Sat, 9 Apr 2011 10:35:56 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
>> On 04/09/2011 10:06 AM, David Seikel wrote:
>>> On Sat, 9 Apr 2011 09:16:56 -0500 Rob Landley
>>> <rlandley at parallels.com> wrote:
>>>
>>>> I need to figure out what infrastructure goes into aboriginal linux
>>>> and what goes into the separate control-images.  (Automative native
>>>> build control images.  Distro bootstrapping.  I suck at naming
>>>> things, anybody have a suggestion?  They can share the mailing list
>>>> until there's enough traffic ot justify splitting it off, but I
>>>> have to name the mercurial repository in order to put it online...)
>>>
>>> Rainbow serpent?  "moves through water and rain, shaping landscapes,
>>> naming and singing of places, swallowing and sometimes drowning
>>> people; strengthening the knowledgeable with rainmaking and healing
>>> powers; blighting others with sores, weakness, illness, and death."
>>
>> Never heard of it before.
> 
> Australian aboriginal mythology that is reasonably common across
> Australia, and quite well known here.

Sigh.

The name "aboriginal linux" is like "aboriginal forests", and has
nothing to do with the native people of a country I've never been to,
who have apparently lost so much of their original culture that they now
identify themselves with a latin phrase ("ab origine") rather than
anything that predated settlement of their continent by europeans, and
have allowed themselves to become so strongly identified that any other
use of that latin phrase is assumed to refer to them, and they're
apparently ok with this in a way that navive americans are not when it
comes to the label "indian".

Hell no.  (And yes I'm getting a touch sick of the topic.)

>>> The control images could be both examples, and bootstrap steps to
>>> the popular distros you have already mentioned.  Stopping short of
>>> being distros or distro builders, and letting the distro maintainers
>>> themselves deal with the next step once we get them bootstrapped.
>>> Though perhaps a full BLFS build is on the cards, since that has a
>>> limit on the packages, is a good shake down, and a good match.
>>
>> Yup.  There's good reasons to _do_ it, it's just the scope of the
>> control images heap could grow larger than the rest of aboriginal
>> linux combined.  How many distros are there?
> 
> That's why I said "popular distros you have already mentioned".  That's
> where you draw the line and say no.  LFS, Gentoo, Debian, Red Hat, and
> Ubuntu.  Maybe ArchLinux as well.  Certainly not Android.

My first exposure to Linux predates Ubuntu by more than a decade, and my
switch to Linux as my exclusive desktop OS predates its first release by
about 7 years.  Arch is noticeably more recent.  I believe Slackware has
more actual users than Linux From Scratch...

Deciding where to stop, and what to include, is policy.  (When users
show up contributing things I wasn't originally interested in, I tend to
take that as a sign there is interest in them.)

>>> Could be both separate, and being able to use Aboriginal Linux as
>>> the build environment by having the two side by side, so people can
>>> run both with simple scripts, then use the results to build some
>>> sort of bootable image, or keep running it under qemu.
>>>
>>> Could also have the option of combining it into one image, and not
>>> two where one is mounted automatically, then have to manually
>>> chroot into it to actually do anything with the bootstrap under
>>> qemu.
>>
>> The control images are target independent, the system images are
>> target specific.  (And the control images can be pretty darn big, the
>> LFS one is 350 megs.)  It makes sense to me to keep them separate.
> 
> Eventually a lot of users of the system will want to build bootable
> images, plus for the likes of LFS, include the toolchain that has been
> used so far.

Yes, that's why it's important to allow them to be used together easily.

I don't see the reasoning for bundling together a target-dependent
automated build that only produces one thing.  Why not just ship the
output instead?  (Which is uploaded tarball you pointed out was misnamed.)

I can see shipping the source, and I can see shipping the end result,
and shipping flexible intermediate stages (such as system images that
can build anything for a target, and control images that can be built
for many different targets).  But why ship an inflexible intermediate
stage that only produces a single result, instead of shipping that result?

>> If you use the run-emulator.sh, it runs the build automatically, but
>> there's a 3 second delay where it prompts you to press a key to get a
>> shell prompt, and only starts the build if you don't.
>>
>> I can easily change how that works, but it's really a user interface
>> question.  (Aesthetic issue, not necessarily a one true answer there.)
> 
> Does it do the chroot step into the bootstrap image when you press
> that key?

Currently you have to go:

  . /mnt/functions.sh
  do_in_chroot /home/lfs /mnt/run-build-stages.sh

Next release (which is horribly overdue and I think I need to back out
the uClibc-NPTL upgrade and put that until the following release just to
get it unblocked), I'm breaking out do_in_chroot to its own script and
putting it in the system image, at which point you should be able to
just go:

  do-in-chroot /home/lfs /mnt/run-build-stages.sh

Also, it should autodetect if /home/lfs is already there, and if so just
--bind mount the host's mounts (/proc /sys /dev and so on) into the
target and do the chroot without re-copying everything.  That requires
some surgery on the script to implement, though...

>>> I'd probably dump a EFL on frame buffer, and EFL on X minimal
>>> bootstrap into the EFL repo, as an example to others to not start
>>> dumping random collections of bootstraps into your repos.  They
>>> already have an OpenEmbedded area in their SVN.
>>
>> I do want to set up a vnc desktop export.  In theory x.org does this
>> built-in, in practice I've misplaced the instructions I got it to work
>> with last time.  (Maybe it was in BLFS somewhere...)
> 
> Or don't pass -nographic to qemu, which I already have a modified
> run-emulator.sh doing.  Something I'll need to play with more for this
> project.

The target's settings file creates the run-emulator.sh script, you can
put anything you want in there.  (Even use an emulator other than qemu.)

However, what I meant was having the image export VNC rather than having
qemu emulate a VNC-based video card.  (Part of the reason is I'm
fiddling with LXC containers in my day job, and I want to be able to
export a VNC desktop from within a container and run it in a window on a
host.  Kir Kolyshkin (OpenVZ maintainer and a co-worker of mine) made
this work at Scale, and I had it working on the Xylinx microblaze last
year, I think the instructions are buried down in BLFS somewhere.  My
recent attempts to google for it have just pulled up external VNC
projects when x.org has built-in support for it these days.  (It's on
the todo list.  The fact my OLS paper proposal "Why containers are
awesome" has been accepted means I need to start seriously researching
this soon...)

> Think I have come up with an acceptable way for me to create a multi
> partition bootable image suitable for DDing onto a CF card, without
> having to use root on the host.

I used to have scripts that did this... (Rummage, rummage...  Oh, duh.)

  http://landley.net/code/mkhda.sh

See also attached control-image script which I whipped up for somebody
on the old list a year or so back.  (Alas, it involved lilo so was
x86-specific, which is why I never followed up on it or merged it.  You
could almost make a universal bootloader out of u-boot, but since it
went GPLv3 it doesn't count anymore.  Since QEMU _has_ a built in
universal bootloader with the -kernel option, I declared victory and
moved on...)

Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: repackage-root.sh
Type: application/x-sh
Size: 2518 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110410/5253a102/attachment.sh>

From onefang at gmail.com  Sun Apr 10 07:31:45 2011
From: onefang at gmail.com (David Seikel)
Date: Sun, 10 Apr 2011 17:31:45 +1000
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <4DA14A08.7040002@parallels.com>
References: <20110408072106.41eadb45@U64-desktop>
	<4D9E52C3.2090908@parallels.com>
	<20110409221237.3ecab345@U64-desktop>
	<4DA06A58.50101@parallels.com>
	<20110410010634.4ca8ef77@U64-desktop>
	<4DA07CDC.50100@parallels.com>
	<20110410015212.24f9a42c@U64-desktop>
	<4DA14A08.7040002@parallels.com>
Message-ID: <20110410173145.0ff43f5b@U64-desktop>

On Sun, 10 Apr 2011 01:11:20 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> On 04/09/2011 10:52 AM, David Seikel wrote:
> > On Sat, 9 Apr 2011 10:35:56 -0500 Rob Landley
> > <rlandley at parallels.com> wrote:
> >>> Could be both separate, and being able to use Aboriginal Linux as
> >>> the build environment by having the two side by side, so people
> >>> can run both with simple scripts, then use the results to build
> >>> some sort of bootable image, or keep running it under qemu.
> >>>
> >>> Could also have the option of combining it into one image, and not
> >>> two where one is mounted automatically, then have to manually
> >>> chroot into it to actually do anything with the bootstrap under
> >>> qemu.
> >>
> >> The control images are target independent, the system images are
> >> target specific.  (And the control images can be pretty darn big,
> >> the LFS one is 350 megs.)  It makes sense to me to keep them
> >> separate.
> > 
> > Eventually a lot of users of the system will want to build bootable
> > images, plus for the likes of LFS, include the toolchain that has
> > been used so far.
> 
> Yes, that's why it's important to allow them to be used together
> easily.

That's what I'm saying.

> >> If you use the run-emulator.sh, it runs the build automatically,
> >> but there's a 3 second delay where it prompts you to press a key
> >> to get a shell prompt, and only starts the build if you don't.
> >>
> >> I can easily change how that works, but it's really a user
> >> interface question.  (Aesthetic issue, not necessarily a one true
> >> answer there.)
> > 
> > Does it do the chroot step into the bootstrap image when you press
> > that key?
> 
> Currently you have to go:
> 
>   . /mnt/functions.sh
>   do_in_chroot /home/lfs /mnt/run-build-stages.sh
> 
> Next release (which is horribly overdue and I think I need to back out
> the uClibc-NPTL upgrade and put that until the following release just
> to get it unblocked), I'm breaking out do_in_chroot to its own script
> and putting it in the system image, at which point you should be able
> to just go:
> 
>   do-in-chroot /home/lfs /mnt/run-build-stages.sh

Ah, so not automated.

> >>> I'd probably dump a EFL on frame buffer, and EFL on X minimal
> >>> bootstrap into the EFL repo, as an example to others to not start
> >>> dumping random collections of bootstraps into your repos.  They
> >>> already have an OpenEmbedded area in their SVN.
> >>
> >> I do want to set up a vnc desktop export.  In theory x.org does
> >> this built-in, in practice I've misplaced the instructions I got
> >> it to work with last time.  (Maybe it was in BLFS somewhere...)
> > 
> > Or don't pass -nographic to qemu, which I already have a modified
> > run-emulator.sh doing.  Something I'll need to play with more for
> > this project.
> 
> The target's settings file creates the run-emulator.sh script, you can
> put anything you want in there.  (Even use an emulator other than
> qemu.)
> 
> However, what I meant was having the image export VNC rather than
> having qemu emulate a VNC-based video card.  (Part of the reason is
> I'm fiddling with LXC containers in my day job, and I want to be able
> to export a VNC desktop from within a container and run it in a
> window on a host.  Kir Kolyshkin (OpenVZ maintainer and a co-worker
> of mine) made this work at Scale, and I had it working on the Xylinx
> microblaze last year, I think the instructions are buried down in
> BLFS somewhere.  My recent attempts to google for it have just pulled
> up external VNC projects when x.org has built-in support for it these
> days.  (It's on the todo list.  The fact my OLS paper proposal "Why
> containers are awesome" has been accepted means I need to start
> seriously researching this soon...)

For this project I'm not using X, just using the kernels frame buffer.
So no VNC for me.

> > Think I have come up with an acceptable way for me to create a multi
> > partition bootable image suitable for DDing onto a CF card, without
> > having to use root on the host.
> 
> I used to have scripts that did this... (Rummage, rummage...  Oh,
> duh.)
> 
>   http://landley.net/code/mkhda.sh
> 
> See also attached control-image script which I whipped up for somebody
> on the old list a year or so back.  (Alas, it involved lilo so was
> x86-specific, which is why I never followed up on it or merged it.
> You could almost make a universal bootloader out of u-boot, but since
> it went GPLv3 it doesn't count anymore.  Since QEMU _has_ a built in
> universal bootloader with the -kernel option, I declared victory and
> moved on...)

That attached script is similar to what I'm doing, only I plan on using
grub legacy instead of lilo.  Unfortunately LFS switched to grub2, so I
have to back track.  I wont be completely to spec with the latest LFS,
but for bootloaders, don't think either of us care.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110410/a9fcaf6e/attachment.sig>

From rlandley at parallels.com  Sun Apr 10 08:52:45 2011
From: rlandley at parallels.com (Rob Landley)
Date: Sun, 10 Apr 2011 03:52:45 -0500
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <20110409192641.3e7d0303@U64-desktop>
References: <20110331180847.1e70e680@U64-desktop>	<4D94771C.7070708@landley.net>	<20110402000443.2caa5c65@U64-desktop>	<4D96BEF5.5040401@parallels.com>	<20110408073606.099e6a7e@U64-desktop>	<4D9E54F1.5030506@parallels.com>
	<20110409192641.3e7d0303@U64-desktop>
Message-ID: <4DA16FDD.30609@parallels.com>

On 04/09/2011 04:26 AM, David Seikel wrote:
> On Thu, 7 Apr 2011 19:21:05 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
>> On 04/07/2011 04:36 PM, David Seikel wrote:
>>> Want me to send you the few BLFS scripts I made for my project?
>>
>> Sure. :)
> 
> Oops, forgot to mention.  The BLFS libjpeg page was last updated in
> 2007, the tarball links they provide no longer work, as the libjpeg
> people have moved on and did not keep the old version around.  So I
> had to bump it two versions.  sigh

The unstable BLFS is much, much more recent than the latest release version:

  http://www.linuxfromscratch.org/blfs/view/svn/

I don't know why they haven't cut a new release in forever, but that
doesn't mean they haven't been updatig the thing.

Rob

 1302425565.0

From rlandley at parallels.com  Sun Apr 10 10:25:26 2011
From: rlandley at parallels.com (Rob Landley)
Date: Sun, 10 Apr 2011 05:25:26 -0500
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <20110410173145.0ff43f5b@U64-desktop>
References: <20110408072106.41eadb45@U64-desktop>	<4D9E52C3.2090908@parallels.com>	<20110409221237.3ecab345@U64-desktop>	<4DA06A58.50101@parallels.com>	<20110410010634.4ca8ef77@U64-desktop>	<4DA07CDC.50100@parallels.com>	<20110410015212.24f9a42c@U64-desktop>	<4DA14A08.7040002@parallels.com>
	<20110410173145.0ff43f5b@U64-desktop>
Message-ID: <4DA18596.5010807@parallels.com>

On 04/10/2011 02:31 AM, David Seikel wrote:
>>>> The control images are target independent, the system images are
>>>> target specific.  (And the control images can be pretty darn big,
>>>> the LFS one is 350 megs.)  It makes sense to me to keep them
>>>> separate.
>>>
>>> Eventually a lot of users of the system will want to build bootable
>>> images, plus for the likes of LFS, include the toolchain that has
>>> been used so far.
>>
>> Yes, that's why it's important to allow them to be used together
>> easily.
> 
> That's what I'm saying.

How is this not already the case?

wget
http://landley.net/aboriginal/downloads/binaries/control-images/lfs-bootstrap.hdc
wget
http://landley.net/aboriginal/downloads/binaries/system-image-i686.tar.bz2
tar xvjf system-image-i686.tar.bz2
cd system-image-i686
./native-build.sh ../lfs-bootstrap.hdc

It helps to install distccd and to download the cross compiler and add
its bin directory to your $PATH (so dev-environment.sh detects it and
sets up distcc in the system image, calling out to the cross compiler on
the host), but that merely speeds things up rather than changing the
outcome.

>>>> If you use the run-emulator.sh, it runs the build automatically,
>>>> but there's a 3 second delay where it prompts you to press a key
>>>> to get a shell prompt, and only starts the build if you don't.
>>>>
>>>> I can easily change how that works, but it's really a user
>>>> interface question.  (Aesthetic issue, not necessarily a one true
>>>> answer there.)
>>>
>>> Does it do the chroot step into the bootstrap image when you press
>>> that key?
>>
>> Currently you have to go:
>>
>>   . /mnt/functions.sh
>>   do_in_chroot /home/lfs /mnt/run-build-stages.sh
>>
>> Next release (which is horribly overdue and I think I need to back out
>> the uClibc-NPTL upgrade and put that until the following release just
>> to get it unblocked), I'm breaking out do_in_chroot to its own script
>> and putting it in the system image, at which point you should be able
>> to just go:
>>
>>   do-in-chroot /home/lfs /mnt/run-build-stages.sh
> 
> Ah, so not automated.

Semi-automated, and working on improving it.  Right now you have to type
two lines, I'm making it so you only have to type one, and documenting
it in the FAQ.

And I suspect "setup-chroot" is a better name for the script...

(Sigh.  FAQ, multiple readmes, the "about" page, documentation.html,
presentation.pdf...  I need to collate it all into one document that
does everything.  Except it's hard for something to be both a tutorial
and a reference and a quick start card.  It's also hard to keep multiple
sources of documentation up to date and in sync, and makes it harder for
people to be sure they've read it all or to know where to start...)

>>> Think I have come up with an acceptable way for me to create a multi
>>> partition bootable image suitable for DDing onto a CF card, without
>>> having to use root on the host.
>>
>> I used to have scripts that did this... (Rummage, rummage...  Oh,
>> duh.)
>>
>>   http://landley.net/code/mkhda.sh
>>
>> See also attached control-image script which I whipped up for somebody
>> on the old list a year or so back.  (Alas, it involved lilo so was
>> x86-specific, which is why I never followed up on it or merged it.
>> You could almost make a universal bootloader out of u-boot, but since
>> it went GPLv3 it doesn't count anymore.  Since QEMU _has_ a built in
>> universal bootloader with the -kernel option, I declared victory and
>> moved on...)
> 
> That attached script is similar to what I'm doing, only I plan on using
> grub legacy instead of lilo.  Unfortunately LFS switched to grub2, so I
> have to back track.

LFS always bought into the idea that GNU software was a good thing.

Personally I treat it like Windows: widely deployed and most people are
already familiar with it, but it's technically utter crap put out by an
organization formed to pursue a non-technical agenda which only does
software as a sideline in service of that other goal.  (One's driven by
profit and the other idology, but neither goal has anything to do with
good engineering.)

> I wont be completely to spec with the latest LFS,
> but for bootloaders, don't think either of us care.

My lfs-bootstrap script is at least one version out of date, I should
update it.

Rob

 1302431126.0

From bjorn.forsman at gmail.com  Sun Apr 10 11:03:25 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Sun, 10 Apr 2011 13:03:25 +0200
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <4DA14A08.7040002@parallels.com>
References: <20110408072106.41eadb45@U64-desktop>
	<4D9E52C3.2090908@parallels.com>
	<20110409221237.3ecab345@U64-desktop>
	<4DA06A58.50101@parallels.com>
	<20110410010634.4ca8ef77@U64-desktop>
	<4DA07CDC.50100@parallels.com>
	<20110410015212.24f9a42c@U64-desktop>
	<4DA14A08.7040002@parallels.com>
Message-ID: <BANLkTi=Ty2Mac3QB4Yoe2jxrzVsRa1E-KA@mail.gmail.com>

On 10 April 2011 08:11, Rob Landley <rlandley at parallels.com> wrote:
[snip]
> You
> could almost make a universal bootloader out of u-boot, but since it
> went GPLv3 it doesn't count anymore. ?Since QEMU _has_ a built in
> universal bootloader with the -kernel option, I declared victory and
> moved on...)

Latest u-boot git still seems to be GPLv2 (according to u-boot/COPYING).
Wolfgang Denk has said that moving to GPLv3 is a long term goal:

http://www.mail-archive.com/u-boot at lists.denx.de/msg16799.html

(near the bottom of the page, search for "Long term goal")

Best regards
Bj?rn Forsman

 1302433405.0

From rlandley at parallels.com  Sun Apr 10 11:35:47 2011
From: rlandley at parallels.com (Rob Landley)
Date: Sun, 10 Apr 2011 06:35:47 -0500
Subject: [Aboriginal] Couple of bugs.
In-Reply-To: <BANLkTi=Ty2Mac3QB4Yoe2jxrzVsRa1E-KA@mail.gmail.com>
References: <20110408072106.41eadb45@U64-desktop>	<4D9E52C3.2090908@parallels.com>	<20110409221237.3ecab345@U64-desktop>	<4DA06A58.50101@parallels.com>	<20110410010634.4ca8ef77@U64-desktop>	<4DA07CDC.50100@parallels.com>	<20110410015212.24f9a42c@U64-desktop>	<4DA14A08.7040002@parallels.com>
	<BANLkTi=Ty2Mac3QB4Yoe2jxrzVsRa1E-KA@mail.gmail.com>
Message-ID: <4DA19613.7010305@parallels.com>

On 04/10/2011 06:03 AM, Bj?rn Forsman wrote:
> On 10 April 2011 08:11, Rob Landley <rlandley at parallels.com> wrote:
> [snip]
>> You
>> could almost make a universal bootloader out of u-boot, but since it
>> went GPLv3 it doesn't count anymore.  Since QEMU _has_ a built in
>> universal bootloader with the -kernel option, I declared victory and
>> moved on...)
> 
> Latest u-boot git still seems to be GPLv2 (according to u-boot/COPYING).
> Wolfgang Denk has said that moving to GPLv3 is a long term goal:
> 
> http://www.mail-archive.com/u-boot at lists.denx.de/msg16799.html
> 
> (near the bottom of the page, search for "Long term goal")
> 
> Best regards
> Bj?rn Forsman

Oh good.  I was going by this:

http://tree.celinuxforum.org/pipermail/celinux-dev/2009-December/001901.html

That's why I never followed up on:

http://tree.celinuxforum.org/pipermail/celinux-dev/2010-October/002056.html

(I note that I am still unlikely to do so, but if somebody ELSE wanted
to...)

I also note that I haven't had a chance to play with this yet either:

http://lists.ozlabs.org/pipermail/devicetree-discuss/2011-March/005112.html

(You may have noticed that my todo list is essentially a giant compost
heap upon which I perform arachaeological digs.)

Rob

 1302435347.0

From rob at landley.net  Tue Apr 12 11:44:17 2011
From: rob at landley.net (Rob Landley)
Date: Tue, 12 Apr 2011 06:44:17 -0500
Subject: [Aboriginal] Splitting up system-image.sh?
Message-ID: <4DA43B11.9080406@landley.net>

(Now that we've got a mailing list back, I'm trying to post to the list
instead of blogging about aboriginal linux design topics.  That way if
you want to respond to any of it, you can easily do so here. :)

I've spent years slowly making the design of this project suck less, via
an incremental series of decisions that generally run like
"mini-native.sh is a stupid name, I should call it root-filesystem.sh.
Having environment variables control what gets built in the root
filesystem is stupid, I should split it into two scripts.  Having a
cross compiler and a native compiler build essentially the same packages
with two different scripts is stupid, I should factor out the common
code such as package builds and make both scripts call them..."

After all that I'm still unhappy with some design decisions, because
I've never been able to come up with a good solution to them.  The "such
and such is ugly/stupid/clearly non-optial" part is easy, but the "I
should do... what?" part.  That's hard.

The system-image.sh kernel build is one of them.  In theory this should
just be another repackaging script like root-filesystem.sh, splicing
together the output of previous stages.  In practice,
SYSIMAGE_TYPE=initramfs needs a just built kernel source directory in
order to bundle a cpio file into the kernel.

Over the years I've put dependency checking into it, with the ability to
detect that the kernel was already there and we didn't need to rebuild
it... and after I stubbed my toe on it breaking a few times I ripped it
back out again as unnecessary complexity that just made the sucker
brittle and hard to maintain and gave me extra configuration stuff to
look up when I wanted to run a test.

  http://landley.net/hg/aboriginal/rev/1163

(Speaking of which: if I can't remember off the top of my head how to
trigger a control, and I wrote it, something is wrong.  Speaking of
which, if anybody has a better suggestion for the name of
$CROSS_HOST_ARCH I'm all ears...)

I'm bumping into this right now because I'm making setup-chroot into its
own script in sources/root-filesystem/sbin and I'm building new system
images to tweak it every time I fix another typo.  (Not testing it on my
host, no.  I had to reinstall my laptop after the first time I ran "make
uninstall" on under buildroot circa 2004 and it suffered host/target
confusion as root while deleting files.  Not going there again.)

The hw- targets are vaguely related to this.  Right now, system-image.sh
isn't quite generic enough.  The kernel build and the root filesystem
image build being tied together like that is ugly and inflexible, and
the _reason_ for it is initramfs.

Back around 2006 I spent a couple weeks trying to figure out how to
replace a symbol out of an ELF file.  You'd think objcopy or something
would be able to do it.  Then I could have the kernel build script
create an ELF vmlinux image, have system-image.sh make a cpio.gz file
(which it basically does internally already) and then splice them together.

Hmmm...  Maybe I can reverse the order.  If I split system-image.sh into:

  1) filesystem-image.sh
  2) kernel.sh
  3) system-image.sh

Then if filesystem-image.sh produces a cpio output file, kernel.sh can
incorporate that when it builds.  (I'd have to copy usr/gen_init_cpio.c
out of the linux kernel source into sources/toys, but I think I'm ok
with that.  I've already got gen_initramfs_list.sh in there and that's
similarly sourced.  Yay GPLv2.)

Thoughts?

Rob

 1302608657.0

From onefang at gmail.com  Tue Apr 12 16:33:48 2011
From: onefang at gmail.com (David Seikel)
Date: Wed, 13 Apr 2011 02:33:48 +1000
Subject: [Aboriginal] undefined reference to `finite'
In-Reply-To: <20110408011719.52c516d1@U64-desktop>
References: <20110405201205.5542b8c9@U64-desktop>
	<4D9AFE66.7040906@parallels.com>
	<20110407232134.09936acb@U64-desktop>
	<20110408011719.52c516d1@U64-desktop>
Message-ID: <20110413023348.08c5a104@U64-desktop>

On Fri, 8 Apr 2011 01:17:19 +1000 David Seikel <onefang at gmail.com>
wrote:

> On Thu, 7 Apr 2011 23:21:34 +1000 David Seikel <onefang at gmail.com>
> wrote:
> 
> > On Tue, 5 Apr 2011 06:35:02 -0500 Rob Landley
> > <rlandley at parallels.com> wrote:
> > 
> > > On 04/05/2011 05:12 AM, David Seikel wrote:
> > > > I'm very tired, and slightly ill, and I can't figure this out.
> > > > 
> > > > One of the packages from the UI library I want to use for this
> > > > project uses a function called finite(double).  As far as I can
> > > > tell, uClibc should supply that function, since we have
> > > > DO_C99_MATH=y in its baseconfig. It's not being found though.
> > > > 
> > > > It's all buried in a forest of #ifdef's that my tired mind is in
> > > > no condition to sort out.
> > > > 
> > > > I'm only two packages away for having all my needed libraries
> > > > installed before I can start cutting code.  Time to sleep.
> > > 
> > > Hmmm...  Posibly a question for the uClibc mailing list?
> > > 
> > > You didn't say which target you were using, so I'll poke at
> > > x86_64:
> > 
> > I'm sticking with doing my builds for 486, since it's still possible
> > that those tiny 486 based boards I looked at last year may be used.
> > If they are not used, then 486 code will at least work fine on
> > whatever we move to.  Any board we move to will be a lot faster
> > anyway, and I've already mentioned that speed is not important for
> > this project.
> > 
> > It appears that finite() is obsolete, and replaced with an
> > isfinite() macro.  A few things I found in my research pointed to
> > that, including the man pages on Ubuntu, but the clincher was this -
> > 
> > http://www.mail-archive.com/uclibc at uclibc.org/msg03404.html
> > 
> > I'll poke the UI library development list about changing that,
> > though the source I'm using is a stable release tarball, but they
> > are thinking of cutting another release soon.  Wont be a major
> > change, I think they only use finite() once.

The UI library upstream have put in a suitable fix now.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110413/015129b6/attachment.sig>

From rlandley at parallels.com  Thu Apr 14 20:03:28 2011
From: rlandley at parallels.com (Rob Landley)
Date: Thu, 14 Apr 2011 15:03:28 -0500
Subject: [Aboriginal] Opinions about reverting uClibc to cut a release?
Message-ID: <4DA75310.9070709@parallels.com>

Getting uClibc-NPTL to work is going to take buckets of debugging (even
x86-64 doesn't work properly for me at the moment), and it's been too
long since the last release.  (Even if this next one hasn't got much in it.)

I could revert in two ways:

1) built the pthreads version of uClibc-NPTL, which is still a
significant regression but not as bad of one.  (I.E. some stuff works.)

2) revert back to uClibc-0.9.31 (resurrecting a half-dozen patches), and
then put uClibc-rc3 back after the release.

I'm leaning towards #1 since it's just a one line config change in
sources/baseconfig-uClibc, but it does mean that LFS doesn't build to
completion on any target anymore.  (I hate these "must step back to make
progress" situations, but switching from pthreads to NPTL is kind of
necessary long-term, and a heck of a lot of pain in the short-term
getting uClibc to pass this kidney stone...)

Rob

 1302811408.0

From onefang at gmail.com  Fri Apr 15 07:46:25 2011
From: onefang at gmail.com (David Seikel)
Date: Fri, 15 Apr 2011 17:46:25 +1000
Subject: [Aboriginal] Opinions about reverting uClibc to cut a release?
In-Reply-To: <4DA75310.9070709@parallels.com>
References: <4DA75310.9070709@parallels.com>
Message-ID: <20110415174625.1aa6b273@U64-desktop>

On Thu, 14 Apr 2011 15:03:28 -0500 Rob Landley <rlandley at parallels.com>
wrote:

> Getting uClibc-NPTL to work is going to take buckets of debugging
> (even x86-64 doesn't work properly for me at the moment), and it's
> been too long since the last release.  (Even if this next one hasn't
> got much in it.)
> 
> I could revert in two ways:
> 
> 1) built the pthreads version of uClibc-NPTL, which is still a
> significant regression but not as bad of one.  (I.E. some stuff
> works.)
> 
> 2) revert back to uClibc-0.9.31 (resurrecting a half-dozen patches),
> and then put uClibc-rc3 back after the release.
> 
> I'm leaning towards #1 since it's just a one line config change in
> sources/baseconfig-uClibc, but it does mean that LFS doesn't build to
> completion on any target anymore.  (I hate these "must step back to
> make progress" situations, but switching from pthreads to NPTL is
> kind of necessary long-term, and a heck of a lot of pain in the
> short-term getting uClibc to pass this kidney stone...)

Number 1 gets my vote, but for no particular reason.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110415/518075ac/attachment.sig>

From alessioigorbogani at gmail.com  Fri Apr 15 07:56:44 2011
From: alessioigorbogani at gmail.com (Alessio Igor Bogani)
Date: Fri, 15 Apr 2011 09:56:44 +0200
Subject: [Aboriginal] Opinions about reverting uClibc to cut a release?
In-Reply-To: <20110415174625.1aa6b273@U64-desktop>
References: <4DA75310.9070709@parallels.com>
	<20110415174625.1aa6b273@U64-desktop>
Message-ID: <BANLkTinjMFjTgOyhvGzgB0e=WsR-_=nsMA@mail.gmail.com>

Hi,

2011/4/15 David Seikel <onefang at gmail.com>:
[...]
>> 2) revert back to uClibc-0.9.31 (resurrecting a half-dozen patches),
>> and then put uClibc-rc3 back after the release.

+1

I would want test *my* unstable code in Aboriginal and not the
unstable code of someone else.

In any case I don't like uClibc at all so my opinion shouldn't be considered.

Ciao,
Alessio

 1302854204.0

From onefang at gmail.com  Sat Apr 16 12:47:45 2011
From: onefang at gmail.com (David Seikel)
Date: Sat, 16 Apr 2011 22:47:45 +1000
Subject: [Aboriginal] ulfs-bootstrap, shrinking LFS.
In-Reply-To: <20110408073606.099e6a7e@U64-desktop>
References: <20110331180847.1e70e680@U64-desktop>
	<4D94771C.7070708@landley.net>
	<20110402000443.2caa5c65@U64-desktop>
	<4D96BEF5.5040401@parallels.com>
	<20110408073606.099e6a7e@U64-desktop>
Message-ID: <20110416224745.40e01038@U64-desktop>

On Fri, 8 Apr 2011 07:36:06 +1000 David Seikel <onefang at gmail.com>
wrote:

> On Sat, 2 Apr 2011 01:15:17 -0500 Rob Landley <rlandley at parallels.com>
> wrote:
> 
> > On 04/01/2011 09:04 AM, David Seikel wrote:
> > > I may just cut iana-etc down to the few ports I actually need for
> > > this project.  The next project will need it for sure.  I've not
> > > actually looked at what iana-etc uses gawk for, but it does smell
> > > of overkill to just create two text files.  Why not just ship the
> > > two text files?
> > 
> > I know.  (Haven't looked at it since october, I think there was some
> > kind of hammer I could hit it with to make it stop...)
> 
> It's on my hit list for this project.

Just a quick look before breakfast.

It looks like iana-etc's purpose in life is to take two files provided
by IANA, and massage them into something that distros tend to prefer.
The two IANI files are provided in the source package, but there are
extra scripts provided to get fresh copies.

For the services file - it adds some "#" comment characters where
needed, and adds aliases from the included port-aliases file (a very
short file with only one actual alias).

The protocols file processing also adds comment characters where needed,
and makes a lower case copy of the keyword to tack at the beginning of
each line of data.  I think this is the only part that needs the
features of gawk.

There is an option to strip the comments out of the results.  The rest
is pretty much installing the files, wrapping the process up in typical
make invocations, and administrivia text files.

I don't see any architectural differences in what happens.  I don't see
any problem with providing a package that has just the resulting files,
plus the license, credits, and a short readme that refers to the source
package. Though perhaps a stripped and an unstripped version?

I might tackle this later today, after breakfast, and if the weekend
social niceties don't get in the way too early.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110416/3e0c64f3/attachment.sig>

From rob at landley.net  Mon Apr 18 12:13:16 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 18 Apr 2011 07:13:16 -0500
Subject: [Aboriginal] Think think think...
Message-ID: <4DAC2ADC.2050607@landley.net>

So over the weekend I poked at doing the split-up of system-image.sh.
Technically it's not hard.

The problem is I'm cluttering the top level directory with build stages.

  http://landley.net/hg/aboriginal/file/110

Way back when (http://landley.net/hg/aboriginal/file/110) we used to have:

  download.sh
  host-tools.sh
  cross-compiler.sh
  mini-native.sh  (became root-filesystem.sh)
  package-mini-native.sh  (became system-image.sh)

Now we have:

  download.sh
  host-tools.sh
  simple-cross-compiler.sh
  cross-compiler.sh
  native-compiler.sh
  simple-root-filesystem.sh
  root-filesystem.sh
  system-image.sh

Now I'd be splitting system-image.sh into:

  image-root-filesystem.sh
  kernel.sh
  system-image.sh

With root-filesystem.sh and system-image.sh both combining existing
stages, I'd be tempted to call the scripts "combine-root-filesystem.sh"
and similar, except that I don't want the result to be called "combine"
and the output directory and the build script are consistently named the
same thing.

Grrr.  Always wanna to do conflicting things...

Ok, I think I need to take the README files and turn them all into HTML
web pages.  For example, http://landley.net/aboriginal/downloads/README
describes the build stage scripts, and who even notices it's there?
(And it's out of date, run-from-build.sh moved into the "more"
subdirectory ages ago...)

Speaking of the "more" subdirectory: I created that to declutter the top
level directory.  It contains things that the user can run but that
build.sh doesn't call.  But I'm not sure that's clear to people, or that
they ever notice it exists...

Sigh.  Documentation, organization for ease of use.  Getting it to
_work_ is hard enough...

Rob

 1303128796.0

From symons.sean at gmail.com  Mon Apr 18 17:30:55 2011
From: symons.sean at gmail.com (sean symons)
Date: Mon, 18 Apr 2011 13:30:55 -0400
Subject: [Aboriginal] does anyone know how to get the m68k prebuilt binary
	to run
Message-ID: <BANLkTimJp4CRL9tAY=JunFretwtSRovD5w@mail.gmail.com>

I haven't been able to get the m68k prebuilt binary to run right, I
found the run-emulator.sh script was blank and put the appropriate
syntax in to run the script and nothing happened. The next step I took
was to proceed to qemu's wiki and used their test ucilinux m68k kernel
which ran with the script but is lacking on the modules required to
mount the squashfs image that is in the m68k prebuilt binary zip i
Downloaded from Landley.net. I may have to move forward and cross
compile my own m68k kernel but before I did I was wondering if anybody
else was able to make this work, thanks in advance.

-Sean Symons-

 1303147855.0

From rlandley at parallels.com  Mon Apr 18 18:03:45 2011
From: rlandley at parallels.com (Rob Landley)
Date: Mon, 18 Apr 2011 13:03:45 -0500
Subject: [Aboriginal] does anyone know how to get the m68k prebuilt
	binary to run
In-Reply-To: <BANLkTimJp4CRL9tAY=JunFretwtSRovD5w@mail.gmail.com>
References: <BANLkTimJp4CRL9tAY=JunFretwtSRovD5w@mail.gmail.com>
Message-ID: <4DAC7D01.4070404@parallels.com>

On 04/18/2011 12:30 PM, sean symons wrote:
> I haven't been able to get the m68k prebuilt binary to run right,

Charles Stevens got it to run under the emulator Aranym, but it involved
a largeish patch to the kernel to add support for aranym's
paravirtualized devices.  (I've cc'd him.)

> I found the run-emulator.sh script was blank and put the appropriate
> syntax in to run the script and nothing happened.

Unfortunately, QEMU only emulates coldfire, which is a nommu m68k subset
that doesn't implement the full instruction set.

People were working on adding full m68k support to qemu:

  http://lists.nongnu.org/archive/html/qemu-devel/2009-05/msg01575.html
  http://gitorious.org/qemu-m68k/qemu-m68k

But I honestly haven't been keeping up...

> The next step I took
> was to proceed to qemu's wiki and used their test ucilinux m68k kernel
> which ran with the script but is lacking on the modules required to
> mount the squashfs image that is in the m68k prebuilt binary zip i
> Downloaded from Landley.net.

Charles told me that the root filesystem worked fine under aranym, which
means the cross compiler and C library worked.  (Although current source
control has uClibc 0.9.32-rc3 which breaks all sorts of stuff and needs
a lot of regression testing with a very large hammer.)

The kernel we're building never worked, he patched one to get it to work
and I could probably dig up his patch if necessary.

> I may have to move forward and cross
> compile my own m68k kernel but before I did I was wondering if anybody
> else was able to make this work, thanks in advance.

Not under qemu. but I haven't tried in over a year.  (I'm happy to
follow up on other people's interest, but really I wanna get _qemu_ to
work, and I'm not up to adding a whole architecture to QEMU which I only
ever wrote one assembly language program for, and that was 15 years ago...)

Rb

 1303149825.0

From rob at landley.net  Mon Apr 18 18:04:08 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 18 Apr 2011 13:04:08 -0500
Subject: [Aboriginal] does anyone know how to get the m68k prebuilt
	binary to run
In-Reply-To: <BANLkTimJp4CRL9tAY=JunFretwtSRovD5w@mail.gmail.com>
References: <BANLkTimJp4CRL9tAY=JunFretwtSRovD5w@mail.gmail.com>
Message-ID: <4DAC7D18.4030605@landley.net>

On 04/18/2011 12:30 PM, sean symons wrote:
> I haven't been able to get the m68k prebuilt binary to run right,

Charles Stevens got it to run under the emulator Aranym, but it involved
a largeish patch to the kernel to add support for aranym's
paravirtualized devices.  (I've cc'd him.)

> I found the run-emulator.sh script was blank and put the appropriate
> syntax in to run the script and nothing happened.

Unfortunately, QEMU only emulates coldfire, which is a nommu m68k subset
that doesn't implement the full instruction set.

People were working on adding full m68k support to qemu:

  http://lists.nongnu.org/archive/html/qemu-devel/2009-05/msg01575.html
  http://gitorious.org/qemu-m68k/qemu-m68k

But I honestly haven't been keeping up...

> The next step I took
> was to proceed to qemu's wiki and used their test ucilinux m68k kernel
> which ran with the script but is lacking on the modules required to
> mount the squashfs image that is in the m68k prebuilt binary zip i
> Downloaded from Landley.net.

Charles told me that the root filesystem worked fine under aranym, which
means the cross compiler and C library worked.  (Although current source
control has uClibc 0.9.32-rc3 which breaks all sorts of stuff and needs
a lot of regression testing with a very large hammer.)

The kernel we're building never worked, he patched one to get it to work
and I could probably dig up his patch if necessary.

> I may have to move forward and cross
> compile my own m68k kernel but before I did I was wondering if anybody
> else was able to make this work, thanks in advance.

Not under qemu. but I haven't tried in over a year.  (I'm happy to
follow up on other people's interest, but really I wanna get _qemu_ to
work, and I'm not up to adding a whole architecture to QEMU which I only
ever wrote one assembly language program for, and that was 15 years ago...)

Rb

 1303149848.0

From rob at landley.net  Tue Apr 19 00:37:07 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 18 Apr 2011 19:37:07 -0500
Subject: [Aboriginal] The hw- target stuff is probably bit-rotted.
Message-ID: <4DACD933.2010807@landley.net>

I'm trying to figure out how much I care.  I myself have never really
used it.  (I've got a Tin Can Tools hammer board I haven't tested this
year, it's on the dining room table under a pile of papers.)  And I
don't regularly hear from other people using it either...

Over the weekend I broke up system-image.sh, factoring out
linux-kernel.sh and root-image.sh.  The third script combines the output
of those two (ala root-filesystem.sh) and also writes the qemu launch
scripts.

I was a bit unsure about having linux-kernel.sh and root-image.sh create
tarballs (not much point, it's one file each and they wind up in
system-image.sh), but build.sh uses 'em as completion indicators, so...
 (I'm not shipping them.)

I also converted one of the READMEs that nobody ever reads to a web
page: http://landley.net/aboriginal/build-stages.html

I need to do the same for the other READMEs (more/README and
sources/README and so on): make 'em all web pages and then maybe people
will actually notice they exist.

I've already got a http://127.0.0.1/aboriginal/downloads/binaries/README
to convert for the binary docs.  And I should really do a page of
documentation on the control images.  (There's a README in
bootstrap-skeleton.  Hands up everybody who'd noticed...)

Rob

 1303173427.0

From onefang at gmail.com  Tue Apr 19 00:48:42 2011
From: onefang at gmail.com (David Seikel)
Date: Tue, 19 Apr 2011 10:48:42 +1000
Subject: [Aboriginal] The hw- target stuff is probably bit-rotted.
In-Reply-To: <4DACD933.2010807@landley.net>
References: <4DACD933.2010807@landley.net>
Message-ID: <20110419104842.4d91a04f@U64-desktop>

On Mon, 18 Apr 2011 19:37:07 -0500 Rob Landley <rob at landley.net> wrote:

> I'm trying to figure out how much I care.  I myself have never really
> used it.  (I've got a Tin Can Tools hammer board I haven't tested this
> year, it's on the dining room table under a pile of papers.)  And I
> don't regularly hear from other people using it either...

If you mean getting an actual bootable disk image out of it like I'm
trying to do now?  So that I can hand it to real hardware soon.

Just wasted a day beating up grub legacy. Think I got it in shape now.
The patches supplied with the version used in the last LFS that used
grub legacy don't actually apply, no matter which patch program you
use.  The advice to set the compiler flags is just plain wrong.  No one
ever actually tested the --disable-nls configure option.  #if
statements refer to symbols that are not defined, instead of using
#ifdef.  I'm sure there was more I can't recall right now.

I seriously think that since that was the last grub legacy that FSF
published, that they went to the effort of breaking it.  But, never
ascribe to malice what can be accounted for with stupidity.

> the control images.  (There's a README in bootstrap-skeleton.  Hands
> up everybody who'd noticed...)

/me puts his hand up.  I've even modified one of the scripts in there
to suit my needs.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110419/6c2dd154/attachment.sig>

From rlandley at parallels.com  Tue Apr 19 01:12:36 2011
From: rlandley at parallels.com (Rob Landley)
Date: Mon, 18 Apr 2011 20:12:36 -0500
Subject: [Aboriginal] The hw- target stuff is probably bit-rotted.
In-Reply-To: <20110419104842.4d91a04f@U64-desktop>
References: <4DACD933.2010807@landley.net>
	<20110419104842.4d91a04f@U64-desktop>
Message-ID: <4DACE184.9030907@parallels.com>

On 04/18/2011 07:48 PM, David Seikel wrote:
> On Mon, 18 Apr 2011 19:37:07 -0500 Rob Landley <rob at landley.net> wrote:
> 
>> I'm trying to figure out how much I care.  I myself have never really
>> used it.  (I've got a Tin Can Tools hammer board I haven't tested this
>> year, it's on the dining room table under a pile of papers.)  And I
>> don't regularly hear from other people using it either...
> 
> If you mean getting an actual bootable disk image out of it like I'm
> trying to do now?  So that I can hand it to real hardware soon.

The hw- targets mostly just let you build a different kernel with a base
architecture's toolchain and root filesystem.  See
sources/targets/hw-tct-hammer/settings for pretty much the only working
example. (The hw-wrt610n one sort of worked, but required a non-default
kernel tarball.  I.E. it only ever built with USE_UNSTABLE=linux and a
provided alt-linux-0.tar.bz2.)

Now that the kernel build is split out, I'm not sure how useful that is.
 You can just build your own and stick it in
"build/linux-kernel-$ARCH".  (The directory expects to have one file,
doesn't matter what it's called.  Ok, don't put a space or quotation
marks in the filename.)

By the way, what's probably bit-rotted is that linux-kernel.sh is
producing output in the $ARCH directory and not the $ARCH_NAME
directory.  And yes, that distinction is too subtle for me to really
want to maintain.  It should all be called $TARGET anyway...

> Just wasted a day beating up grub legacy. Think I got it in shape now.
> The patches supplied with the version used in the last LFS that used
> grub legacy don't actually apply, no matter which patch program you
> use.  The advice to set the compiler flags is just plain wrong.  No one
> ever actually tested the --disable-nls configure option.  #if
> statements refer to symbols that are not defined, instead of using
> #ifdef.  I'm sure there was more I can't recall right now.

Yeah, that's the FSF.  As with many packages, it used to work, then it
was contributed to the FSF and it gradually bloated and became incoherent.

> I seriously think that since that was the last grub legacy that FSF
> published, that they went to the effort of breaking it.  But, never
> ascribe to malice what can be accounted for with stupidity.

So back up a few versions.

>> the control images.  (There's a README in bootstrap-skeleton.  Hands
>> up everybody who'd noticed...)
> 
> /me puts his hand up.  I've even modified one of the scripts in there
> to suit my needs.

Heh.

I'm converting it all to HTML so more people notice, but my day job is
unhappy and making them happy again is a higher priority on my time
right now...

Rob

 1303175556.0

From rob at landley.net  Tue Apr 19 18:23:12 2011
From: rob at landley.net (Rob Landley)
Date: Tue, 19 Apr 2011 13:23:12 -0500
Subject: [Aboriginal] Temporarily reverting uClibc to get a release out.
Message-ID: <4DADD310.2000107@landley.net>

Ok, commit 1329 back in February updated uClibc to -rc2, and then 1330
changed the baseconfig, 1338 swapped around the patches, 1339 bumped the
version to -rc3, and 1340 switched everything from setjump/longjmp stack
unwinding to dwarf (because NPTL requires it).

NPTL doesn't even build for x86-64 for me because it's trying to suck
non-PIC binary code into a shared library.  I'm not sure why this works
for other people, haven't tracked it down yet.  But nobody on the list
was interested in the issue:

  http://lists.uclibc.org/pipermail/uclibc/2011-March/045020.html

Switching back to pthreads makes most of hte targets work, but powerpc
is still broken, dunno why.

Attempting to build LFS under the result (on target i686) runs into fun
little issues like:

  http://lists.uclibc.org/pipermail/uclibc/2011-March/045021.html

To which the uClibc developers basically said the fact that this package
used to build unmodified against uClibc, and no longer does, is not
something they consider a regression.

Uh-huh.

So for right now, I need to untangle those 5 commits above, put the old
uClibc back, and move the -rc series into the USE_UNSTABLE slot so I can
keep testing/fixing it no matter what its upstream developers think.
(And the fact that the packages are alt-uClibc but the trigger is
USE_UNSTABLE=uClibc is an annoyance I need to resolve.  Consistent
naming is a good thing, hard to even talk about it otherwise...)

Rob



 1303237392.0