[Toybox] config directive vs commands with underscores

[Rob Landley]

On 04/14/2012 01:35 PM, Elie De Brauwer wrote:
> Hello list,
> 
> I bumped into a little problem. I was working on the addition of
> pivot_root, which is actually just a 1-to-1 mapping to a system call.
> 
> However I ran into a problem with the "config PIVOT_ROOT" statement.
> This translates itself in the build searching for a file called pivot.c
> and nog pivot_root.c. In toysh.c there are already some directives with
> TOYSH_something but I'd like some pointers on how to properly deal with
> this situation (or how to workaround it, e.g. put the source in
> pivotroot.c but still have the application called pivot_root but I don't
> know how the config directives and the NEWTOY macro's are coupled.

Yeah, I hit this with nbd-client.c and switch_root.c.

I need to change the sed thing to use the first argument to newtoy as
the filename to look for, and not try to get it from the config symbol.

I wrote up a big long design rant about this, but didn't post it for the
longest time because it didn't come to any _conclusions_, just described
the current status quo and not what it _should_ do. It looks like I
never actually hit "send".  Hmmm...

Ah, here's the unfinished file:

-----------------------------

Devise, devise... (big long design digression about id/groups/whoami)

Musing out loud over a design decision while I work out the best way to
implement shared infrastructure to handle a new corner case.

In the pending patch pile, I've got three commands in the same file, two
of which are trivial wrappers for the third.  (Basically whoami is "id
-un" and groups is "id -Gn").  I merged a different whoami
implementation, but a wrapper might still be smaller if id already has
that code, and it's nice to be _able_ to do this anyway.

Previously, I had things like "nc" and "netcat" being _aliases_ for the
same command, or "sh" and "toysh". And having two names for the same
command works, but that's not the same a wrapper.

These new commands behave _differently_, and I'd like these different
commands to call different main() functions and have different help
entries, but live in the same file. Right now, it doesn't do that.

Currently at the start of each file there's one or more magic lines:

  USE_CMDNAME(NEWTOY(cmdname, "options", FLAGS))
  USE_CMDNAME(OLDTOY(cmdname, mainprefix, "options", FLAGS))

Plus some kconfig text:

  config CMDNAME
    bool "cmdname"
    help
      This is the help text

>From the above, we need to get the following information (mostly
described in http://landley.net/toybox/code.html#generated ):

1) The kconfig plumbing for defconfig and menuconfig.

This is the generated/Config.in kconfig stuff so each CONFIG_CMDNAME to
be separately enabled or disabled.

Note that generated/Config.in is used to create .config (that's what
"make defconfig" does), and then generated/config.h (the CFG_CMDNAME and
USE_CMDNAME() macros) is created from .config later in the build.

(And yes, busybox uses both the CFG_ and USE_() stuff, because I
introduced them when I maintained the project. The fact they've turned
back into #ifdef salad since I left is one of the reasons I didn't go back.)

2) The gcc command line, so the right toys/cmdname.c files eventually
get included on the compiler command line, which is worked out in
scripts/make.sh.

3) Help text for the help command. We just use the kconfig help text for
this, although the eventual plan is to not just concatenate sub-options,
but also merge "usage:" lines.  For example, if you look at toys/sort.c
it's got CONFIG_SORT and CONFIG_SORT_BIG, which means we have two usage
lines:

  usage: sort [-run] [FILE...]
  usage: sort [-bcdfiMsz] [-k#[,#[x]] [-t X]] [-o FILE]

Which ideally would become:

  usage: sort [-runbcdfiMsz][-k#[,#[x]] [-t X]] [-o FILE] [FILE...]

And is more likely to just become:

  usage: sort [-bcdfiMsz] [-k#[,#[x]] [-t X]] [-o FILE] [-run] [FILE...]

Right now scripts/config2help.py is doing that (but only for the top
level config symbols, no sub-symbols or usage: line mergin), and I've
got half of scripts/config2help.c written that will do it without
requiring python on the build system.  (This is why my tarballs contain
generated/help.h for defconfig, and why that file is in "make distclean"
but not "make clean". Toybox should build on systems that haven't got
python. Note that development != build, your dev machine has X11
installed but your compile farm might not. By the way, I screwed this up
for the 0.2.1 release, so you need python to build it.  Sigh.)

where was I...

4) generated/newtoys.h

This is just all the "magic lines" above concatenated together (and
sorted in alphabetical order so I can binary search structures generated
from them).  Then various definitions of NEWTOY() and OLDTOY() can be
#defined in order to:

4A) Provide the function prototypes for all the cmdname_main() functions
in toys.h

4B) Fill out the toy_list[] array (from main.c).  This is why newtoys.h
is run through sort when it's created: the array is binary searched, so
must be in alphabetical order.

4C) define the NEED_OPTIONS constant so the option parsing
infrastructure can drop out at compile time if nothing uses it (also
from main.c.  Mostly this comes up if you build a single command: not
all of them need the option parsing infrastructure.)

I'm glossing over a few bits like DEFINE_GLOBALS() (used for
generated/globals.h), see http://landley.net/toybox/code.html if you
want the full thing.

The _problem_ I'm having with id/groups/whoami I is that the
USE_COMMAND(NEWTOY(command, "options", FLAGS)) has some redundant
information (COMMAND and command have to match), and some of the fields
are used to mean several different things:

1) the toys/cmdname.c part of the compiler command line is _actually_
generated from the USE_CMDNAME bit. It chops out the text after the
first underscore, squashes it to lower case, and that's your
toys/$BLAH.c name.  This is probably wrong, but it's got to match up
with the CONFIG_CMDNAME in .config with the filename so it can tell what
to include and what not to.  Since the USE_CMDNAME macro is derived from
the CONFIG_CMDNAME, they've got to match.  The text at the start of
newtoy() doesn't quite.  (It's really there so you can do mixed-case
command names, if you really need a "uC" command and such.)

2)

-----------

Basically the problem is the block of data at the top of each command
has redundant information, and this is a problem, and I need to fix it.
 But since the various chunks of it currently serve different purposes,
I'm not quite sure what a less-redundant version should look like.

This post was related:

http://lists.landley.net/pipermail/toybox-landley.net/2012-February/000048.html

And it came up again in the flags thing.  (I'd love to be able to use
__FILE__ but unfortunately that's got the path in it, and the build
paths are all "toys/command.c" when what I want is just "command".

I'm willing to do clever things with with the C preprocessor, but would
rather not run source files through sed or m4 before compiling them.
Unfortunately, the C preprocessor is "stone knives and bear skins"
1970's technology: it doesn't really do string handling, you can't have
a macro #define a new symbol...

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.