[Aboriginal] Refactoring/recycling

[Tristan Van Berkom]

On Fri, 2016-09-09 at 19:28 -0500, Rob Landley wrote:
> I've been breaking aboriginal linux down for parts. It's just too
> much
> of a giant hairball, despite my attempts to separate it into layers.
> 

Hi Rob,

So just to understand, since I have some interesting work based on this
by now, what is going to be the end result of using the musl-cross-make 
approach in Aboriginal ? How much will things change from the
perspective of the Aboriginal end user ?

If I understand correctly, this will mean:

  o GPLv3 toolchain

  o Less deviation from upstream gcc (as I understand there had been
    some patches to make gcc behave differently for aboriginal, while
    the musl-cross-make patches are mostly upstreamed)

Are you planning to continue with ccwrap.c, allowing for the
relocatable cross compiler ? (I think this was a pretty cool feature,
although I understand it's painful to keep up with gcc here).

In the end, will we have essentially the same bootstrapping procedure ?

I think there is a high value in depending on a minimal amount of host
tools, building toybox with host tools and isolating the build is great
for repeatability and reliability of the build across multiple host
OSes.

On a side note, I wonder if there is knowledge we can borrow from the
openembedded project - particularly where it comes to targeting the
compiler for a specific arch, configuring the kernel for that arch and
putting the qemu invocation together.

See:
  http://git.openembedded.org/openembedded-core/tree/meta/conf/machine/

And the include directory for fun.

Honestly there are a variety of reasons why I prefer using an LFS type
of build (but automated) based on aboriginal over using the yocto
approach of cross compiling all the way up the stack, but I think
openembedded have come a long way in terms of configuring the base
kernel/compiler/qemu and it would be great to leverage that. It also
has a similar approach to aboriginal here (inasmuch as it uses a
machine conf, like aboriginals target files).

Best Regards,
    -Tristan

> The thing is, once you take out the toolchain build (and replace it
> with
> Rich Felker's musl-cross-make), there's only 5 other packages it
> builds
> (toybox, busybox, make, bash, distcc). All of them compile reasonably
> easily, and as toybox adances at least 2 of them should go away. The
> infrastructure aboriginal linux provides for them is WAY OVERKILL for
> what's left for it to do (as evidenced by
> https:/github.com/j-core.org/mkroot working fine without any of it).
> 
> What Aboriginal Linux has is:
> 
> 1) The ./download.sh infrastructure to download and patch source
> code,
> and maintain an extracted "package cache" for parallel builds. This
> includes setup and teardown wrappers to _use_ the package cache (and
> set
> up the cross compiler path, and so on).
> 
> 2) The host-tools.sh "airlock step" and environment sanitization
> logic
> (sources/variables.sh) for doing "hermetic builds" (I.E.
> portable/reproducible builds that provide their own known build
> environment).
> 
> 3) target configs with matching gcc tuple (and flags), kernel config
> (and KARCH), and qemu command line (and kernel command line).
> 
> 4) Wrapper script to launch qemu with some infrastucture around it
> (most
> of which supports build control images; note that
> http://landley.net/aboriginal/build-control to _create_ build control
> images is a separate project with its own git repo).
> 
> 5) Busybox and uClibc config files (now legacy info).
> 
> 6) A dozen package build recipes (in sources/sections), 2/3 which
> (ccwrap.sh, gcc.sh, musl.build, uClibc.build, binutils.build,
> linux-headers.sh, uClibc++.build, elf2flt.sh) are obsolete if we use
> Rich's toolchain instead of building our own.
> 
> 7) The miniconfig.sh stuff
> 
> 8) record-commands.sh
> 
> I'm still reading through my old stuff, but that seems to be the bulk
> of it.
> 
> If you move to musl-cross-make and mkroot.sh, #5 and #6 can pretty
> much
> be abandoned. I can extend mkroot.sh to (optionally) build make and
> distcc for a target, neither is a big deal. I need to get serious
> about
> toybox's built-in shell.
> 
> I can break out #1 into its own project. I hadn't yet because it
> seems
> like overkill: mkroot doesn't do it and hasn't needed it? Still, Jeff
> was hitting the lack of this one last week...
> 
> I'm already porting #2 (the host-tools.sh stuff) into the toybox
> build.
> Maybe there should be a "scripts/hermetic.sh" to set up a hermetic
> build
> environment, maybe it should be an install target, not sure yet.
> (It's
> not _just_ an install, there's a chroot-like step to change the $PATH
> and unset all unrecognized environment variables checking the
> whitelist...) The record-commands.sh stuff should should go in there
> too, it's separate but related.
> 
> I should throw miniconfig at the kernel again and see if it sticks.
> (It
> won't, but as far as I'm concerned reposting it with explanation
> annually is probably good enough.)
> 
> That leaves #3 and #4 which I can maybe wrap up together as their own
> thing? Haven't decided about that bit yet. Maybe it belongs alongside
> mkroot?
> 
> Rob
> _______________________________________________
> Aboriginal mailing list
> Aboriginal at lists.landley.net
> http://lists.landley.net/listinfo.cgi/aboriginal-landley.net