[Toybox] [PATCH] Add makefile rule to build kconfig; fixes clean-tree parallel builds

[enh]

On Tue, Mar 3, 2026 at 3:11 PM Rob Landley <rob at landley.net> wrote:
>
> On 2/27/26 16:07, enh wrote:
> > On Sun, Feb 22, 2026 at 6:49 PM Rob Landley <rob at landley.net> wrote:
> >>
> >> On 2/13/26 13:36, enh wrote:
> >>>> Anyway, in theory, android could now just do something like:
> >>>>
> >>>> KCONFIG_ALLCONFIG=scripts/android_miniconfig scripts/genconfig.sh -n &&
> >>>> scripts/make.sh
> >>>>
> >>>> And... it might work? (Don't have the test environment.)
> >>>
> >>> ...you had me worried here that i was going to have to change
> >>> something, but it seems like the update "just worked"? is that
> >>> expected, or have i missed something and just haven't noticed the
> >>> other shoe fall yet?
> >>
> >> Commit 32decfacef12 is probably the other shoe, you'll have to at least
> >> resnapshot the headers because the probed symbol names changed.
> >
> > yeah, that's fucked it :-)
>
> Alas.
>
> > here's what we're doing on each update (at the root of our toybox clone):
> >
> > ```
> > rm -rf .config generated/ android/
> >
> > function generate() {
> >    which=$1
> >    echo -e "\n-------- $1\n"
> >
> >    # These are the only generated files we actually need.
> >    files="config.h flags.h globals.h help.h newtoys.h tags.h"
>
> Hmmm... You're missing generated/Config.in which seems kind of important.

these are just the files needed by the android build system to
actually build toybox. (though maybe i should just copy all the files
so future maintainers don't have to think/do anything if you add extra
generated files...)

> $ make clean
> cleaned
> $ scripts/genconfig.sh
> $ ls generated/
> Config.in  help.h  unstripped
> $ scripts/make.sh
> Compile toybox
> ...
> $ ls generated/
> build.sh  Config.in  globals.h  newtoys.h  unstripped
> config.h  flags.h    help.h     tags.h     zhelp.h
>
> Running scripts/make.sh won't run scripts/genconfig.sh (anymore), so it
> won't recreate generated/Config.in.
>
> (If you don't enable gunzip you won't get zhelp,

yeah, i've thought about that on and off ... given that we check these
files in, until/unless someone demonstrates a significant saving i've
been erring on the side of "readable blob". (though one day i should
probably at least do the experiment and see how much space compression
would save.)

> "unstripped" is a
> directory that gets mkdir -p by make.sh, and build.sh is more or less
> documentation that gets harvested for the prereq build. The rest you need.)
>
> >    cp config-$which .config
>
> ok.
>
> >    NOBUILD=1 scripts/make.sh
>  >    out=android/$which/generated/
> >    mkdir -p $out
> >    for f in $files; do cp generated/$f $out/$f ; done
> >    rm -rf .config generated/
>
> My first guess is to add Config.in to the list at the top.
>
> That said, the build no longer compiles and runs test binaries, and all
> the inputs to sed/grep etc that generate those headers are checked into
> the repo, with the single exception of the "cc -dM -E -" output, which
> is checked for the presence or absence of two #defines. Historically
> "config" had a lot more dependencies than "make", but I'm trying hard to
> fix that.
>
> The other stuff that used to be probed got moved to #ifdefs and
> __has_include() and so on, but configuration dependencies are resolved
> before then so anything that "depends on SYMBOL" needs a symbol.
>
> There are actually only six probed "depends on" left: three for
> IS_ANDROID (log/sendenvent/mkpasswd) and three for IS_FORK (all in
> pending, and they can just break the build in certain configs until they
> get fixed up, that's part of what needing to be fixed up MEANS.)
>
> All the OTHER uses of those symbols could just as easily use an if
> (IS_ANDROID) or if (HAS_FORK) which could geet set in portability.h via
> appropriate #ifdeffery and not involve the config system at all (except
> as a strict consumer of its output).
>
> I already intend to clean up mkpasswd. It would just be REALLY nice if
> ONE WEEK would go by without a fresh
> https://mstdn.jp/@landley/116134666430118741 and me looking at jobs as a
> nurse's assistant or something. (Unionized! Excellent health insurance!
> Not having to care about being fucked over by Gavin Newsom _too_!)
>
> I just built "sendevent" on glibc, my linux headers have the ioctl
> define and nothing else really looks android specific? (I mean nothing's
> probably LISTENING to the ioctl, but the tool doesn't care.) So there
> doesn't seem to be a reason not to remove that "depends on" now? (It's
> no more out of place in defconfig than deallocvt or i2cdump.)

yeah, sgtm.

> "log" really looks like a reinvention of posix's logger. I could
> probably work up a small shim so a non-bionic build of it would
> basically else fall back to syslog(). (I'm not translating buffers to
> facilities, but the syscall can do what it likes with the input.)

yeah, no reason not to --- android actually ships both log(1) and
logger(1) so callers can use the extra android bits if they want, or
write something portable if they want. (sadly log(1) already existed
before i took this over, otherwise i'd have just added extra
android-specific arguments to logger(1).)

> If all the inputs of scripts/genconfig.sh were confirmed to be
> checked-in files, and we didn't parse compiler output anymore, would
> THAT be good enough to run scripts/genconfig.sh as part of the android
> build?

i'm skeptical that the cost of maintaining a parallel set of genrules
in the android build system would ever be worth the cost.

i think the two best outcomes are either:

1. we end up in a world where everything is __has_include() or equivalent

2. you add a script that does the "take a config file and output the
generated files" all in one, so that the stuff that changed here is
all "behind the scenes" and we always just have the [import time]
one-liner --- not because calling _two_ scripts is hugely onerous, but
because _two_ scripts isn't an interface in the same way a single
script is :-)

world 2 seeming more realistic because there would still be the "aye,
but which toys do you actually want to build?" question with world 1.
though i suppose that could just be a set of -Ds that either comes
from the build system or from a generated include file.

but like i say, other than just adding a one-stop "black box" entry
point, i don't think there's much value to any work here.

> Sigh. I'm aware that mac (and cygwin?) builds suck.

corp security nonsense prevents me from actually checking right now,
but iirc we fixed mac?

./scripts/portability.sh:: ${CPUS:=$(($(nproc 2>/dev/null || sysctl -n
hw.ncpu 2>/dev/null)+1))}

> Would a "forkbomb"
> parallel build of the prereq stuff help? I believe this is posix-2008:
>
> for i in lib/*.c $FILES
> do
>    X=${FILES##*/} X=${X%.c}
>    $BUILD -c $i -o $X.o &
> done
> wait
> $BUILD $LINK *.o -o toybox-prereq && rm *.o || exit 1
>
> (The hard part of parallel builds is the rate limiting. If you just want
> to launch EVERYTHING in parallel then wait and link... well that. Sigh,
> I need a way to comment out the last line of build.sh so use.sh could do
> the parallel one instead. "Simple" a moving target when you add
> features...)
>
> >    make allnoconfig KCONFIG_ALLCONFIG=config-$which
> > }
> >
> > generate "device"
> > generate "linux"
> > generate "mac"
> >
> > rm -rf .config
> > ```
>
> I'd take a look at those input configs, but if android's toybox is still
> public and up to date anywhere outside google's firewall I haven't found
> it. :(

the AOSP tree is moving forward, albeit only as infrequent snapshots.
until this update the last change was last summer anyway, so you'd
already have that:

commit d9ce38ab5e1b7e1fa8250dc5fe3be1e4719262da
Author: Elliott Hughes <enh at google.com>
Date:   Wed Jul 2 12:41:57 2025 -0700

    Remove obsolete configuration symbols.

    Warnings from post-update.sh:
    ```
    bad symbol ID_Z
    bad symbol MKDIR_Z
    bad symbol MKFIFO_Z
    bad symbol MKNOD_Z
    bad symbol SORT_FLOAT
    ```

    Change-Id: If23abf40b28054f6e8477e5c29f9adb9df2c7c3c

> > (and, yes, i can confirm that it's
> > 32decfacef12644c8c939466db58f1370ccba4d0 that broke things, not the
> > more recent change.)
>
> There used to be a less strict separation between config and make, so
> make.sh would recreate generated/Config.in.
>
> > it seems like my "config" file is being used (because i get an error
> > until i rename CONFIG_TOYBOX_ON_ANDROID to CONFIG_IS_ANDROID) but then
> > when i look at the resulting generated/config.h file it's wrong. for
> > example:
> >
> > generated/config.h:#define CFG_TOYBOX_SELINUX 0
> >
> > despite
> >
> > CONFIG_TOYBOX_SELINUX=y
> >
> > in my config...
>
> That's... odd.
>
> I still don't have the input .config you're using to reproduce it here,
> but one thing that might be screwing you up is that scripts/make.sh now
> has this near the start:
>
> # Run oldconfig if necessary
> [ -e "$GENDIR"/Config.in ] ||
>    KCONFIG_ALLCONFIG="${KCONFIG_ALLCONFIG:-$KCONFIG_CONFIG}" \
>    scripts/genconfig.sh -d || exit 1
>
> So the .config you copied in place before running make.sh may get
> stomped by the oldconfig before you've got your ducks in a row? (The
> theory is you run scripts/genconfig.sh _then_ run scripts/make.sh, but
> you're doing it in reverse order. That "make allnoconfig" is a call to
> scripts/genconfig.sh.)
>
> Try doing this:
>
> $ make distclean
> $ KCONFIG_ALLCONFIG=your_config_file scripts/genconfig.sh -n
> $ NOBUILD=1 scripts/make.sh
>
> Does that create the generated/ files you want? (It does here...)
>
> $ KCONFIG_ALLCONFIG=scripts/android_miniconfig scripts/genconfig.sh -n
> $ NOBUILD=1 scripts/make.sh
> warning: using unfinished code from toys/pending
> $ grep SELINUX .config generated/config.h
> .config:CONFIG_TOYBOX_SELINUX=y
> generated/config.h:#define CFG_TOYBOX_SELINUX 1
> generated/config.h:#define USE_TOYBOX_SELINUX(...) __VA_ARGS__
> generated/config.h:#define SKIP_TOYBOX_SELINUX(...)

yeah, this seems to work (i've deleted the cp because i think that was
already unnecessary because we're specifying which file to use as the
input config):

diff --git a/post_update.sh b/post_update.sh
index 2b42dba9..66b23009 100755
--- a/post_update.sh
+++ b/post_update.sh
@@ -21,14 +21,12 @@ function generate() {
   # These are the only generated files we actually need.
   files="config.h flags.h globals.h help.h newtoys.h tags.h"

-  cp config-$which .config
+  KCONFIG_ALLCONFIG=config-$which scripts/genconfig.sh -n
   NOBUILD=1 scripts/make.sh
   out=android/$which/generated/
   mkdir -p $out
   for f in $files; do cp generated/$f $out/$f ; done
   rm -rf .config generated/
-
-  make allnoconfig KCONFIG_ALLCONFIG=config-$which
 }

 generate "device"


thanks!

> Rob