[Aboriginal] miniconfig-busybox

[Rob Landley]

On 12/01/2011 02:43 PM, James McMechan wrote:
> 
>>> I saw David Seikel's comments about still being interested in trim-config
>>> Here is how I use mini-config with busybox, it is not hard
>>> though I have not yet gotten the auto build stuff to include the aboriginal
>>> tarball and the package directory in my initramfs
>>
>> There was an initramfs bug recent-ish-ly, are you using a version after
>> http://landley.net/hg/aboriginal/rev/1366 ?
> 
> I was using Aboriginal-1.1.0 but am having a few problems
> I seem to remember it working fine with 1.0.1, 1.0.0 and 0.9.11
> now for some reason the patch command is failing, I will need to double
> check the config file for busybox it may have a new knob in config somewhere

I fixed the toybox/busybox patch command in August and again in October,
possibly you're hitting one of those bugs.

I need to get the next release out, but I keep hoping I can beat some
sense out of sparc first.  (It's almost building LFS!  Progress!)

>>> --- busybox.orig 2011-11-27 05:41:26.000000000 -0800
>>> +++ busybox.build 2011-11-27 16:19:20.000000000 -0800
>>> @@ -5,7 +5,8 @@
>>> # Build busybox
>>>
>>> #make allyesconfig KCONFIG_ALLCONFIG="${SOURCES}/trimconfig-busybox" &&
>>> -make defconfig &&
>>> +#make defconfig &&
>>> +make allnoconfig KCONFIG_ALLCONFIG="${SOURCES}/mini-busybox" &&
>>
>> I need to provide a way to select this, do a defconfig if no busybox
>> config is provided, otherwise use that. Probably some kind of config
>> setting.
> 
> well there is the classic method to try all three,

Yeah, but should it expect to find them in $SOURCES (where all the other
files would be checked into source control, but these wouldn't) or at a
location specified in config?  The overlay mechanism for extra root
filesystem files is config based for the same reason, mixing tracked and
untracked files in the same directory is untidy.

Long-term I want to move to toybox and just not HAVE buckets of
extraneous crap needing to be disabled...

> there are prettier layouts though
> done=0
> [ $done -eq 0 -a -e "${SOURCES}/mini-busybox" ] && make allnoconfig KCONFIG_ALLCONFIG="${SOURCES}/mini-busybox" && done = 1
> [ $done -eq 0 -a -e "${SOURCES}/trim-busybox" ] && make allyesconfig KCONFIG_ALLCONFIG="${SOURCES}/trim-busybox" && done = 1
> [ $done -eq 0 ] && make  defconfig && done = 1
> [ $done -ne 0 ] && \

Probably an if/else staircase.  That's what they're for.

>> I'm also back to working on toybox, which means testing toybox, which
>> means getting toybox and busybox to install alongside each other and
>> gradually reduce the number of busybox commands I'm using.
> 
> Personally, I like building up from the bottom, rather than trying to trim
> the fat from the top...

When you're migrating a system that currently works by replacing
components, you have to build the new thing while dismantling the old,
so you wind up doing both...

> On a side note, I noticed busybox defconfig even has a webserver so
> you could add your own private package mirror to dev-environment.sh

I could.  But I prefer to leave that sort of thing to other people: I'm
intentionally agnostic about what you _do_ with the resulting system.
The hdc infrastructure lets you make self contained build images, but my
examples include one that just runs the busybox test suite.

>> I'm not inflicting toybox on aboriginal's default configuration any time
>> soon, but I personally need the option.
>>
>>> cp .config "$WORK"/config-busybox &&
>>> LDFLAGS="$LDFLAGS $STATIC_FLAGS" make -j $CPUS $VERBOSITY $DO_CROSS &&
>>> make busybox.links || dienow
>>>
>>> I was adding very little to config
>>>
>>> export NO_CPLUSPLUS=1
>>> export SYSIMAGE_TYPE=initramfs
>>>
>>> My preference is for mini-config rather than trim-config so I can turn on just what I want
>>
>> I admit the trimconfig was a bit of a hack; trying to use defconfig but
>> it didn't work. And now busybox defconfig is back to not working
>> because the UBI stuff doesn't build on ubuntu 11.10.
> 
> I like miniconfig primarily because of the lack of changes, if some later version needs a extra
> config option the option is ignored by previous versions, the base stuff in busybox mostly
> just works.  It is the advanced stuff and new options that usually give me grief.

Busybox had a 1.0 years ago and was presumably feature complete at that
time.  Several of its commands weren't feature-complete enough to work
in a build environment, but very few were actually _missing_.

Since then it's grown about 100 commands.

>>> I am using a ~98 line mini-config which needs few changes between busybox versions
>>> It is possible that a few more options can be turned off I would like to convert the
>>> initramfs to bz2 and drop gzip.
>>> Without genext2fs it appears it doesn't need TAR_OLDGNU_COMPATIBILITY
>>> apparently that program's build environment was very very old.
>>
>> I need to write my own genext2fs. Now that I'm back working on toybox,
>> I may manage that eventually...
> 
> I usually just use initramfs I only need ~2 device files to boot
> for testing so the setup for cpio is quite easy even by hand.

For /dev I'm just mounting -t devtmpfs these days, as of commit 1414 in
July.  If you only have a root user in the emulated system, that's plenty.

> mdev from busybox takes care of the rest of the device files later.
> The new DEVTMPFS in the kernel might make even that unneeded
> I have not tested without /dev/console yet though...

Doesn't devtmpfs give you /dev/console?

>>> #needed for od -t option in busybox build
>>> CONFIG_DESKTOP=y
>>
>> I've argued with Denys about that. CONFIG_DESKTOP is not well defined,
>> there's no clear rule for what it does and doesn't do.
>>
>> This config is enough to build aboriginal linux itself, with this in
>> hostconfig?
> 
> It was working earlier, last tested about 1.0.2, I need to check with the new
> version.
> 
>> A) Does that include running the ./download.sh stage?
> 
> download.sh does not use the host tools path and breaks at the moment

It will if you run host-tools.sh, zap packages, and then re-run
./download.sh.

But yeah, that's not the normal order of operations, which is why I've
learned to test that separately. I only occasionally run the aboriginal
build itself under a system image (to catch that sort of thing), and
it's usually download.sh that breaks if something's going to.

> I did not set wget or ifconfig in busybox nor test download.sh

Note that sources/root-filesystem/sbin/init.sh has two paths twoards the
end, one for PID 1 and one for being called in a chroot.

>> B) Have you tried lfs-bootstrap.hdc with this?
> 
> I have not been checking LFS I was looking more like tinycore linux
> and most of my testing has been under user-mode-linux since I had
> that ready to run already

I mention it because lfs-bootstrap.hdc automates building LFS 6.8, so
that's my biggest test case right now.

  http://landley.net/aboriginal/control-images

> When testing I repeatedly run the same kernel with different initramfs
> or recompile the kernel repeatedly with the same initramfs they are
> conceptually separate in my mind so I almost always work on one
> or the other alternately.
> 
> I noted that the command I have been testing with does not seem to
> work for host tools like might be hoped.
> STAGE_NAME=host-tools more/test.sh i686 build_section busybox
> it gets all unhappy about pod2text pod2man & pod2html, but

Busybox always gets unhappy about those when perl isn't installed, but
it's not a fatal error.

> leaves a statically linked busybox binary in host-tools-i686

host-tools-i686?

Um, host-tools.sh is target agnostic, it imports sources/include.sh but
never calls load_target.  I'm confused?

> It would be somewhat easier if host acted like a real target.

It's sort of the point of the host stuff is to be target agnostic.  You
should be able to build all the targets under the same host environment.
 (The fact that different releases of Ubuntu and Fedora and such don't
_provide_ the same host environment is why host-tools.sh exists, but the
point stands.)

> The current special case logic seem to get in my way as much as help
> when trying repeated host-tools setups.

Which special case logic?

I broke the initialization in half so include.sh sets the basic
variables and then load_target does all the target-specific stuff.  So
the "special case is" that host-tools.sh and download.sh are _not_ doing
the target initialization.

Now there's a bunch of special case crap for "on some versions of ubuntu
the 'gcc' in the path is a perl script, we need to unwrap that", and
"you have distcc installed so we have to care about more than one
instance of 'gcc' in the path", and so on.  But that's because various
host environments we run on are funky.

> It was also quite awkward to use the default simple-cross-compiler.sh,
> when I was having trouble with uClibc or linux-headers as it kept
> rebuilding binutils & gcc.

This is why I broke the package builds into sources/sections and made
the build_section function to call them, and also why I added
more/test.sh to run snippets in build context (with sources/include.sh
and load_target called):

  http://landley.net/aboriginal/FAQ.html#debug_test

Its "usage:" example suggests setting STAGE_NAME so in this case you
probably want:

STAGE_NAME=simple-cross-compiler more/test.sh sparc build_section uClibc

I should probably put that example in the FAQ...  Done.

> I quickly commented them out, but it would
> seem that having each package saved in the build directory not just
> the assembled meta-packages would be nice.

Um, do you mean like the config option BINARY_PACKAGE_TARBALLS?

> For example busybox-host.tbz busybox-i686.tbz uClibc-host.tbz
> 
> My opinion is when a system is working well simple things are
> simple, and hard things are straightforward. I almost seems like
> too much has been spent here make the hard parts easy and the
> simpler parts have gotten more complex as a result.

That's always a danger.

I agonized about putting in the package cache, and wound up writing
EXTENSIVE DOCUEMNTATION on it (it's about half the FAQ).  But the fact was:

A) it sped up the build _tremendously_, especially when building
multiple architectures on a machine with a finite amount of memory.

(And even for single builds you could wind up extracting the kernel
tarball _four_times_ during the build: kernel headers for simple cross
compiler, cross compiler, and native compiler, and then actually
building the kernel binary.  Even busybox is in host-tools _and_
simple-root-filesystem...)

B) You can conceptually completely ignore it, and pretend "setupfor
blah" is just a wrapper for "mkdir build/blah && tar xvjfC
packages/blah.tbz build/blah && cd build/blah" and that cleanup is "cd
.. && rm -rf blah".

I wouldn't have done A without B.  I _didn't_ do A until I could figure
out how to make B true.

Rob

 1322832882.0