[Aboriginal] Build from version control system repos instead of from tarballs

[Rob Landley]

Sorry for the delay answering, thunderbird is totally melting down. I
left it trying to check email for 12 hours and it failed. I can see this
in the web archive, but not in my email.

> Hi All,
>
> Can Aboriginal grabs sources from git(Linux, musl, uclibc, busybox)
> and hg(toybox) repos instead of from tarballs?
> Thanks!

Yes, I do it for testing all the time, but the procedure's a bit
strange. The way I do it is:

  mkdir -p build/packages
  hg clone http://landley.net/hg/toybox build/packages/alt-toybox
  USE_ALT=toybox ./build.sh i686

Which is easy to do, but takes some explaining to understand what's
going on here:

1) USE_ALT says to use an alternate version of this package. This is a
whole separate parallel version of a source package, with separate
tarball, package cache, and patches. (Same build scripts though.)

A) The tarball (if any) is downloaded from the ALT= in download.sh (and
stored as packages/alt-$PACKAGENAME-0.tar.bz2 in case you're wondering).

B) The package cache for it is build/packages/alt-$PACKAGENAME

C) Patches (if any) are applied from
sources/patches/alt-$PACKAGENAME*.patch files.

You can build alternate versions of more than one package at a time by
comma separating them, ala USE_ALT=busybox,toybox,linux .

(You can even have getconfig spit out a different config file for alt
stuff, ask me about that if you care. It falls back to spitting out the
normal one by default, and 99% of the time that's all I need these days.)

2) If there's no tarball in packages and no URL to download it from, but
the extracted package cache entry in build/packages exists, download.sh
leaves it alone.

So by specifying "use the alt- version of this package" and then
supplying the package cache version of it myself using a source control
checkout, I can test more or less behind the build system's back. This
works because for most packages I don't have an ALT= URL specified (or
it's commented out), and I haven't got any alt- patches checked into the
repo.  (When I do need them they're generally just symlinks to the
release version patches.)

If you _do_ have patches, the setupfor function notices that there's no
sha1-for-source.txt in the package cache version, assumes it hasn't been
patched yet, and applies the patches (listing the sha1sums it applied
there, but no tarball sha1sum when there isn't one). If you take
advantage of this, you can do a "git clean -fdx && git checkout -f" or
similar in the package cache to undo this before your next bisect step,
and it'll try to reapply them next build. You can set
ALLOW_PATCH_FAILURE=1 if you don't mind patches failing to apply, but
mostly for testing I just break down and "HOST_EXTRA=perl
./host-tools.sh" (or put the symlink in build/host by hand) so I don't
need to apply the perl removal patches while bisecting a kernel bug.

Does that answer your question?

I've thought about teaching the build to check a "repositories"
directory at the top level (next to "packages") and use the source out
of there if it finds it, but what do I do about patches then? (Apply the
ones the tarball takes? Never apply them and have it be the user's problem?)

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.