[mkroot] To the people who keeping pinging me off list...

[David Seikel]

On Tue, 13 Mar 2018 13:28:33 -0500 Rob Landley <rob at landley.net> wrote:

> On 03/13/2018 12:04 PM, David Seikel wrote:
> > I've only glanced at mkroot, and it seems to be similar to parts of
> > Aboriginal Linux, so I expect I'll be able to use it easily.
> 
> The general idea is hermetic (I.E. mkroot) doesn't build its own
> toolchains, and with that moved out the remaining aboriginal linux
> infrastructure was way too heavyweight for what it did, so I started
> over.
> 
> Rich Felker (the musl-libc maintainer) created his own toolchain build
> (https://github.com/RichFelker/musl-cross-make) using current packages
> (gcc/binutils, musl, etc). And I gave him a lot of feedback along the
> way (no the kernel headers in /usr/include/linux aren't optional,
> please add native compilers), and the result was something usable.
> 
> I make an mcm-buildall.sh script to build all the cross and native
> compilers musl-cross-make currently supports (that I know of). I'd
> like to push that script upstream but Rich wants to do things his own
> way. :)
> 
> The new build is intentionally minimalistic. The basic root
> filesystem build is "mkroot.sh" by itself (which I should rename
> build.sh this weekend when it becomes hermetic). Additional build
> scripts can go in modules/ to compile more packages, the most
> interesting of which is "kernel" to build a linux kernel. You can
> copy mkroot.sh and modules/kernel.sh into their own directory and run
> them by themselves, they have no external dependencies except bash
> and some standard commands out of the $PATH. Eacy is about 200 lines
> of bash.
> 
> The airlock step is now part of toybox (make install_airlock). If you
> look in toybox's scripts/install.sh you'll see:
> 
> # The following are commands toybox should provide, but doesn't yet.
> # For now symlink the host version. This list must go away by 1.0.
> 
> PENDING="bunzip2 bzcat dd diff expr ftpd ftpget ftpput gunzip less
> ping route tar test tr vi wget zcat awk bzip2 fdisk gzip sh sha512sum
> unxz xzcat bc"
> 
> # "gcc" should go away for llvm, but some things still hardwire it
> TOOLCHAIN="ar as nm cc make ld gcc objdump"
> 
> The TOOLCHAIN line is the toolchain binaries the build needs in the
> host $PATH. The first are the remaining things we copy out of the
> host $PATH which toybox should provide.
> 
> The PENDING list is stuff toybox should be providing, but isn't yet.
> About half of it's in the pending directory but hasn't been cleaned
> up and promoted yet.
> 
> Hermetic is still building a busybox binary, mostly for the command
> shell. (Doing a proper toybox shell is the single largest remaining
> todo item. Doing ask and vi are fighting for second place.) That
> busybox binary has:
> 
> 	bunzip2, bzcat, bzip2, gunzip, gzip, hush, ping, route, sh,
> tar, unxz, vi, wget, xzcat, zcat
> 
> Which I believe is entirely encapsulated by the "pending" list, above.
> 
> I've been chipping away at the $PENDING list, but should probably
> context switch to getting the native toolchains plugged in. At the
> moment this involves adding a modules/make.sh because I haven't got
> make in toybox yet. (It's another biggish one. Less big than awk,
> probably around about sed's complexity.)
> 
> In theory the native toolchain should just extract and run, but I
> need to figure out how to package it. (Aboriginal had a squashfs and
> some symlinks, I may do something similar. It's too big to go in the
> cpio.gz with the rest of the filesystem, it extracts tond  half a
> gigabyte.)
> 
> Right now I've been getting kernel+initramfs to build and boot on as
> many qemu targets as possible, with one block device. Back under
> aboriginal build control images needed _three" block devices (root
> filesystem, writeable /home, source and build scripts. When the root
> filesystem moved to initramfs, /dev/hda because the toolchain
> squashfs, still requiring three block devices.)
> 
> I could package them together into a partitioned image, but all three
> partitions kinda want to be independent at a design level (native
> toolchain, writeable scratch space, build control image.)
> 
> I'm considering trying to do a network filesystem instead? The SMB 3.0
> protocol's posix mode sounds sane (more so than nfs or older smb;
> alas 9p is basically dead). I have a todo item to make an smb3 server
> in toybox and have that available for mounting on the host. (And if
> you loopback mount a file from the smb server, it's not going to care
> what's crazy about the actual protocol...)
> 
> I could also try to use qemu's built-in virtfs, but it's 9p based and
> kinda crazy-ish. (Requires extra host libraries during the build
> because the qemu guys thing extended attributes require a shared
> library to use, so most people won't have it enabled...)
> 
> Or I could just go back to "let's mount 3 block devices". (Maybe I
> can use virtual USB disks? Except then it's hard to control what
> order they go in, and each serves a different function and needs a
> unique identifier...)
> 
> So there are still some design decisions to make here. :)

Sounds like my goal of having mkroot produce a 486 LFS 8.2 bootable
image by the end of next weekend might not be such a good idea if you
are gonna change it next weekend anyway.  I'll switch to 486 Aboriginal
Linux + LFS for this week then, and switch to mkroot / hermetic later
when the dust settles.  So I get to play with your lfs-bootstrap control
image after all.  The LFS build scripts I have been writing should slot
into that easily enough anyway, they are using similar concepts.  Common
infrastructure to get and unpack the package tarballs, then call build
scripts named after the packages to actually build them.  LFS has it's
own airlock step, chapter 5.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.landley.net/pipermail/mkroot-landley.net/attachments/20180314/16a71874/attachment-0001.sig>