From rob at landley.net  Sat May 18 06:46:18 2013
From: rob at landley.net (Rob Landley)
Date: Sat, 18 May 2013 01:46:18 -0500
Subject: [Aboriginal] Fwd: Re: [Tinycc-devel] ARM hardfloat prolog
	[daniel-gl@gmx.net]
Message-ID: <1368859578.18069.261@driftwood>

If you were wondering why big endian arm is so hard to get working:

QEMU doesn't currently support it.

Rob
-------------- next part --------------
An embedded message was scrubbed...
From: Daniel =?iso-8859-1?q?Gl=F6ckner?= <daniel-gl at gmx.net>
Subject: Re: [Tinycc-devel] ARM hardfloat prolog
Date: Wed, 1 May 2013 21:14:08 +0200
Size: 5662
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20130518/d0645571/attachment.eml>

From rob at landley.net  Sat May 18 06:02:56 2013
From: rob at landley.net (Rob Landley)
Date: Sat, 18 May 2013 01:02:56 -0500
Subject: [Aboriginal] Busybox applets not yet implemented
In-Reply-To: <14647.50.0.225.8.1366353383.squirrel@lavabit.com> (from
	idunham@lavabit.com on Fri Apr 19 01:36:23 2013)
Message-ID: <1368856976.18069.257@driftwood>

Cleaning out my drafts folder, I have this:

On 04/19/2013 01:36:23 AM, idunham at lavabit.com wrote:
> Landley wrote:
> > I've thought of installing busybox in one directory and toybox in
> > another and having the $PATH fall back between them, but busybox is
> > going away. Toybox is replacing busybox. So the extra plumbing would
> > just be transitional.
> >
> > build/host/busybox currently says:
> >
> > Currently defined functions:
> > 	[, [[, ash, awk, bunzip2, bzcat, bzip2, cpio, dd, diff, egrep,
> > expr,
> > 	fgrep, find, ftpd, ftpget, ftpput, grep, gunzip, gzip,
> > ifconfig, init,
> > 	install, less, lspci, man, mount, mv, pgrep, ping, pkill, ps,
> > route,
> > 	sed, sh, split, stat, tar, test, tr, umount, unxz, vi, wget,
> > xzcat,
> > 	zcat
> >
> > vi and less need the same line editing code as the shell command
> > history, otherwise reasonably straightforward.
> >
> > grep/egrep/fgrep are mostly just regex code wrappers, I should do  
> that.
> 
> > mount/umount I started and got interrupted, need to get back to it.
> 
> My own experiments with booting a Debian system with a custom shell  
> init
> indicate that only a few things are needed to boot up a system  
> properly:
> -a sh: pd/mksh is enough for now.

I have a lot of design work done for toysh. Wack-a-moleing my way to  
it...

> -mount: the next most important part.

That's fairly high on my list, I just keep getting toys/pending/* files  
I need to clean up before doing new stuff...

> -mdev: fixing /dev does demand some subdirs, and hotplug will require
> handling env variables (which busybox has support for);
> not highest priority, though.

Question: how much does devtmpfs handle (there was a recent thread on  
busybox about an environment variable they weren't using listing  
subdirectories)

> -grep, modprobe, cut: I autoload modules by
> grep -h MODALIAS /sys/bus/*/devices/*/uevent 2>/dev/null |cut -d = -f  
> 2
> |xargs modprobe -abq

Should there be a tool that does this? (An mdev flag, perhaps?)

> modprobe needs the following for this use case:
> Support aliases. May require depmod.
> -a: insert all modules in a list at once
> -b: blacklist needs to work!
> -q: Hot/coldplug shouldn't spit out a mile of errors.

Um, blacklist? (Explain? Why is this needed, and the man page says  
there's a config file...?)

> > cpio and tar are fairly straightforward, on my todo list.
> 
> Of the two, tar may be more important for building, but cpio is what
> matters for replacing klibc-utils.

I'm pondering doing a kickstarter/indiegogo to see if anybody's  
employers want to throw money at me to work on this stuff full-time for  
a while. (Alas, I'm no longer in a position to take half a year off and  
just live off savings. I got a real house and the bills that go with  
it...)

> > man is a design issue: not sure what it _should_ do. A trivial zcat  
> of
> > preformatted text is trivial, it should integrate with the "help"
> > command somehow. (help documents shell builtins, man documents stuff
> > installed in $PATH, for toybox both are builtin but man should also
> > check $PATH for external commands. But _not_ the horrible troff  
> crap.)
> 
> Standard man often caches formatted troff as text, which would be a  
> good
> place to start.

Yup. But that doesn't help the whole "build from source, rebuild from  
source under the result" self-bootstrapping thing. It still requires a  
"real system" containing external tools that are not part of this  
system.

You can't get rid of the old stuff if you still need it. Thus I still  
consider this an unsolved problem... (todo item.)

> > ash and sh are a biggish deal, I've got a lot of research for that  
> but
> > need a couple months to do it right. expr is basically the $((blah))
> > shell builtin. [, [[, and test are all the same command, which I've
> > started a local implementation of.
> 
> There's plenty of shells around, so I'm not too concerned about it.
> 
> Of course, this is all my own opinion, looking at it from the boot
> perspective.

I've gotten very little done the past month because I've been swamped  
by day job work, but I finally got a chunk of it checked in so a little  
of the pressure's off, at least for the weekend. (I'm also helping a  
friend move so I don't get the whole weekend off, but still. I can at  
least catch up a bit on email. :)

Rob

 1368856976.0

From rob at landley.net  Sat May 18 05:40:26 2013
From: rob at landley.net (Rob Landley)
Date: Sat, 18 May 2013 00:40:26 -0500
Subject: [Aboriginal] LFS 7.3 toybox issue
In-Reply-To: <20130415195143.GA11873@desktop> (from prazzb@gmail.com on Mon
	Apr 15 14:51:43 2013)
Message-ID: <1368855626.18069.256@driftwood>

Sorry, I've had this in my drafts for a month. (Got buried by dayjob,  
finally taking a weekend to catch up a bit...)

On 04/15/2013 02:51:43 PM, Prasanna Balan wrote:
> Hello
> I am struck in package shadow as it needs chmod -f. However chmod -f
> isn't yet implemented in toybox. So I just pushed it above in
> baseconfig-busybox. However, it still had the same issue. On further
> search, I found that busybox is building chmod but toybox is
> overwriting it.

Did I forget to switch off chmod in busybox? Ah, no I didn't. The  
"stuff in toybox" comment is used to indicate the point at which to  
chop the file. From sources/sections/busybox.build:

   getconfig busybox | sed '/toybox/,$d' > "$WORK/config-busybox"

Note that if you run "build/host/busybox" it won't list chmod.

> Then it dawned that toybox doesn't parse baseconfig-busybox but rather
> does defconfig.

Toybox is gradually replacing busybox in aboriginal. Once I get a  
toybox command working, I switch off the corresponding busybox command.  
(You can set BUSYBOX=1 in config to use disable the toybox build and  
busybox defconfig instead.)

> Is there a reason why it is done this way or such issue never came
> up??

Because toybox is replacing busybox as I get commands implemented.

> Should I just patch toybox to do chmod as default n ???

No, I just patched toybox to support -f.

Rob

 1368855626.0

From idunham at lavabit.com  Sun May 19 07:22:29 2013
From: idunham at lavabit.com (idunham at lavabit.com)
Date: Sun, 19 May 2013 00:22:29 -0700
Subject: [Aboriginal] Busybox applets not yet implemented
In-Reply-To: <1368856976.18069.257@driftwood>
References: <14647.50.0.225.8.1366353383.squirrel@lavabit.com>
	<1368856976.18069.257@driftwood>
Message-ID: <20130519072229.GA903@Caracal>

On Sat, May 18, 2013 at 01:02:56AM -0500, Rob Landley wrote:
> Cleaning out my drafts folder, I have this:
> 
> On 04/19/2013 01:36:23 AM, idunham at lavabit.com wrote:
> >Landley wrote:
> >My own experiments with booting a Debian system with a custom
> >shell init
> >indicate that only a few things are needed to boot up a system
> >properly:
> >-a sh: pd/mksh is enough for now.
> 
> I have a lot of design work done for toysh. Wack-a-moleing my way to
> it...
> 
> >-mount: the next most important part.
> 
> That's fairly high on my list, I just keep getting toys/pending/*
> files I need to clean up before doing new stuff...
 
> >-mdev: fixing /dev does demand some subdirs, and hotplug will require
> >handling env variables (which busybox has support for);
> >not highest priority, though.
> 
> Question: how much does devtmpfs handle (there was a recent thread
> on busybox about an environment variable they weren't using listing
> subdirectories)

I'll have to reboot and check that...Ok, very little.
These are all the subdirs in /dev when devtmpfs is the only tool used
but all modules are loaded:

/dev/bus/usb
/dev/bus/usb/005
/dev/bus/usb/005/003
/dev/bus/usb/005/002
/dev/bus/usb/005/001
/dev/bus/usb/004
/dev/bus/usb/004/001
/dev/bus/usb/003
/dev/bus/usb/003/001
/dev/bus/usb/002
/dev/bus/usb/002/001
/dev/bus/usb/001
/dev/bus/usb/001/001
/dev/dri/card0
/dev/dri/controlD64
/dev/bsg/0:0:0:0
/dev/input/mouse1
/dev/input/mouse0
/dev/input/mice
/dev/cpu/1
/dev/cpu/1/cpuid
/dev/cpu/1/msr
/dev/cpu/0
/dev/cpu/0/cpuid
/dev/cpu/0/msr

Note that I use OSS4, so the sound subsystem is not typical.

DEVNAME can include any number of subdirs, so be aware of that.
And the issue was that busybox was not getting $DEVNAME on coldplug
(mdev -s); the solution was to open the proper uevent file and read 
DEVNAME from there.

> >-grep, modprobe, cut: I autoload modules by
grep -h MODALIAS /sys/bus/*/devices/*/uevent 2>/dev/null |cut -d = -f 2|xargs modprobe -abq
> 
> Should there be a tool that does this? (An mdev flag, perhaps?)

That's possible, but I tend to think that something I can do in 92
characters and one line of shell is not a justification for a new
feature--features that are that duplicative of standard tools sound like
a GNU project. ;-)
(Admittedly, grep -h is not POSIX.  But you'll need it for the kernel
build.)

Also, if you do add an mdev flag, it should be something that you can
run separately from seeding /dev; autoload MUST take place AFTER
manually loading modules, or you complicate things like loading Linux-PHC
modules (where an out-of-tree module provides a needed hook for an
in-tree module or builtin). I've fought with this a good bit. 

> >modprobe needs the following for this use case:
> >Support aliases. May require depmod.
> >-a: insert all modules in a list at once
> >-b: blacklist needs to work!
> >-q: Hot/coldplug shouldn't spit out a mile of errors.
> 
> Um, blacklist? (Explain? Why is this needed, and the man page says
> there's a config file...?)

-b turns on respecting the blacklist config file.
The default is "ignore blacklists"; this is reasonable because you will
sometimes want to test what happens, and you usually have a reason for
loading a kernel module manually.  In other words, assume the user knows
what they're doing.
But if you autoload modules, you will run into the situation where some
module that's autoloaded crashes the computer, or messes up the sound
(look at /etc/modprobe.d/alsa-base-blacklist.conf), or something of like
sort. It may be a bug, but it happens, and nothing is more annoying than
a bug you know the cause of but don't have the tools to workaround.

So there are two real options:
1. Assume the administrator knows what he's doing, and load anything
unless you tell modprobe to be careful.  Scripts can use modprobe -b and
the admin can use modprobe.
2. Assume that whoever wrote the blacklist usually knows better than the 
admin. Don't load blacklisted modules unless the admin forces it, with 
modprobe -f; scripts can just use modprobe.

Option 2  optimizes for shorter scripts at the expense of longer command 
lines and more troubleshooting.

Option 1 is what standard modprobe uses.

The config "file" for blacklisting is /etc/modprobe.d/*.conf, formerly
/etc/modprobe.conf.
/etc/modules has stuff to autoload.

HTH,
Isaac Dunham


 1368948149.0

From patrick at gentoo.org  Fri May 31 00:29:30 2013
From: patrick at gentoo.org (Patrick Lauer)
Date: Fri, 31 May 2013 08:29:30 +0800
Subject: [Aboriginal] Replacing bash with mksh
Message-ID: <51A7EEEA.8050004@gentoo.org>

Hi everyone,

Just for fun (and to see how everything works) I've patched the build
scripts and successfully built the Aboriginal images with mksh instead
of bash. Because we can, and because bash 2 is so old and crufty.

mksh is the MirBSD Korn Shell ( https://www.mirbsd.org/mksh.htm ), it
has an active upstream, a nice license and has proven portability to
lots of weird architectures and operating systems.

The attached patch shows some not-so-good assumptions in the package
name and version handling. Took me a while to figure out why the tarball
was "not found" :)
And I disabled distcc building because I had some really weird build
failures that I haven't tracked down yet.

Have fun,

Patrick
-------------- next part --------------
A non-text attachment was scrubbed...
Name: aboriginal-mksh.patch
Type: text/x-patch
Size: 3050 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20130531/c9e2ee85/attachment.bin>

From metaniklas at gmail.com  Fri May 31 08:00:09 2013
From: metaniklas at gmail.com (Niklas Larsson)
Date: Fri, 31 May 2013 10:00:09 +0200
Subject: [Aboriginal] Replacing bash with mksh
Message-ID: <51a858b6.0a04980a.6ba1.ffff9ac6@mx.google.com>

Another nice thing with mksh is that it's the default shell in Android since Ice cream sandwich.

----- Ursprungligt meddelande -----
Fr?n: "Patrick Lauer" <patrick at gentoo.org>
Skickat: ?2013-?05-?31 02:25
Till: "aboriginal at lists.landley.net" <aboriginal at lists.landley.net>
?mne: [Aboriginal] Replacing bash with mksh

Hi everyone,

Just for fun (and to see how everything works) I've patched the build
scripts and successfully built the Aboriginal images with mksh instead
of bash. Because we can, and because bash 2 is so old and crufty.

mksh is the MirBSD Korn Shell ( https://www.mirbsd.org/mksh.htm ), it
has an active upstream, a nice license and has proven portability to
lots of weird architectures and operating systems.

The attached patch shows some not-so-good assumptions in the package
name and version handling. Took me a while to figure out why the tarball
was "not found" :)
And I disabled distcc building because I had some really weird build
failures that I haven't tracked down yet.

Have fun,

Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20130531/2d8bcdf3/attachment.htm>

From rob at landley.net  Fri May 31 13:39:30 2013
From: rob at landley.net (Rob Landley)
Date: Fri, 31 May 2013 08:39:30 -0500
Subject: [Aboriginal] Experimenting with mksh as shell for aboriginal
In-Reply-To: <51A6E62A.1080703@star-group.net> (from
	patrick.lauer@star-group.net on Thu May 30 00:39:54 2013)
Message-ID: <1370007570.2776.52@driftwood>

On 05/30/2013 12:39:54 AM, Patrick Lauer wrote:
> On 05/30/2013 12:51 PM, Rob Landley wrote:
> > On 05/29/2013 12:20:51 AM, Patrick Lauer wrote:
> >> There's a few little glitches that I noticed, so maybe I'll either
> >> document that or throw patches at you - in the meantime here's a  
> little
> >> funny hack: Replacing bash with mksh, because old versions suck ;)
> >
> > Old versions do indeed suck.
> >
> > My current plan is to implement toysh in toybox as a full bash
> > replacement, and migrate to that. But mksh is less obsolete than  
> bash 2.
> And it has an active upstream that tests on weird platforms ...

Which is nice, but I'm trying to get the number of separate packages in  
the base system down. I originally planned to replace bash with one of  
the busybox shells (then Bruce happened), now I plan to do toysh but  
have a bunch of commands like "mount" to implement first...

So I'm looking at this as "fun, but stopgap". That said, I've  
downloaded the original public domain version and might loot cleaned up  
versions of some of its code for toysh in toybox...

> >> It doesn't work completely reliable yet, and I had some weird  
> distcc
> >> issues along the way (which is why it's commented out)
> >
> > Did you notice the control-images stuff?
> >
> > http://landley.net/aboriginal/control-images
> >
> > Specifically the lfs-bootstrap.hdc although I think I need to check  
> in
> > a couple tweaks and rebuild the binary. (I have a directorty for
> > updating it from LFS 6.8 to 7.3. It's on my todo list. :)
> >
> > It's not just aboriginal rebuilding itself,
> Looks like fun, let's see if I find some time for playing around with  
> it ...

I need to find some time to update it. :)

> >
> >> But as you can see it's a rather invasive change in a dozen lines  
> of
> >> code - you built a good foundation.
> >
> > Thanks. :)
> >
> > Mind if I forward this to the mailing list?
> I've figured out why it failed (your noversion() function is too
> pessimistic), let me send a working patch later today/tomorrow. I've
> managed to build almost all targets (cc implodes on armv4eb and armv6,
> will try to figure out what happens there ...) and boot them (sh4  
> fails
> for me, looks like either an invalid kernel image or qemu-1.4 being  
> buggy).

armv4eb isn't fully supported because qemu still hasn't quite got  
support for it (or so the qemu mailing list keeps saying), so I haven't  
really tested it.

Last I checked armv6 was working, though.

Oh, sh4 needs a patched qemu. (The following patch whitespace-damaged  
by cut and paste.)

--- a/hw/char/sh_serial.c
+++ b/hw/char/sh_serial.c
@@ -248,11 +248,9 @@ static uint64_t sh_serial_read(void *opaque,  
hwaddr offs,
                      s->flags &= ~SH_SERIAL_FLAG_RDF;
              }
              break;
-#if 0
          case 0x18:
              ret = s->fcr;
              break;
-#endif
          case 0x1c:
              ret = s->rx_cnt;
              break;

> See you on the mailing list,
> 
> Patrick
> 

I should just cc: it here.

Rob
 1370007570.0