[Toybox] help text

[Rob Landley]

On 2/1/20 11:48 PM, enh wrote:
> On Fri, Jan 31, 2020 at 6:58 PM Rob Landley <rob at landley.net> wrote:
>> There should probably be a documentation section on all this, just dunno if it's
>> in code.html, design.html, FAQ.html, cleanup.html... (Maybe there should be a
>> docs.html I try to shuffle all of it into? Hmmm...)
> 
> i certainly always have trouble with
> http://landley.net/toybox/code.html#lib_args --- the ^ and ? that you
> explained recently come _after_ a section on globals. it might be
> helpful if it was on a page on its own, so it's clear where the
> relevant stuff starts and ends.

Well, it is. The problem is that page is the big comment at the start of lib/args.c.

  https://github.com/landley/toybox/blob/master/lib/args.c#L12

> (or just better html formatting of
> that page. if you're paying enough attention, there are the filenames
> there already. but i've misread that page numerous times.)
> 
> more examples might help too. your explanation of ^ versus ? on the
> list was more helpful.

The best I've got so far is use in existing commands:

  grep 'NEWTOY(.*"[?^]' toys/*/*.c

> or, to your point here, yes, maybe a "how to get the command-line
> parsing and help right" how-to type document would be more useful for
> that purpose than the current "read the source walkthrough, which
> actually serves multiple purposes".

I'm thinking youtube videos, I just haven't gotten around to it because $DAYJOB.
(I just got approved for a 5 year residence card thing for tokyo! Woo!)

>>> (btw, am i the only one who thinks the busybox --list output format is
>>> more useful than the toybox one?)
>>
>> toybox hasn't got a --list, it treats any unrecognized -- as --long. (Leftover
>> from back in the days I was trying to squeeze out every byte and going "the
>> multiplexer is in every command". I should probably fix that since "simple"
>> implying "consistent" has risen above "small" in the goal list. Assuming the
>> change won't break somebody's install script...)
>>
>> Note that "busybox --help" and "busybox" produce the same output, but "toybox
>> --help" and "toybox" don't. The busybox without arguments version is
>> gratuitously hard to parse; it's got --help info followed by a list with commas
>> which screw up "for i in $(toybox); do echo $i; done" even if you chop off the
>> right magic number of leading lines.
>>
>> I explained this to Denys when I met him in person at CELF in 2010
>> (http://lists.busybox.net/pipermail/busybox/2010-April/072078.html) and he added
>> a --list that produced script consumable output saying in the commit comment it
>> was because I asked him to:
>>
>>   https://git.busybox.net/busybox/commit/?id=5a7c72015c05
>>
>> Space vs newline is the same as far as most consumers are concerned (the above
>> for loop doesn't care), and fitting more on the screen seemed easier on humans
>> to me?
>>
>> That said, adding a "type --help" prompt to "toybox walrus" might be useful, or
>> just specifically checking for long so "toybox --list" tells you to run "toybox
>> --help". (If you really want --list to be ignored and thus produce the
>> toybox-with-no-args output, I can do that? Or do you think the one command per
>> line version is superior...?)
> 
> i suspect i'm not the only one who didn't know that the existing
> `toybox` output is usable in something like `for i in $(toybox) ...`,
> whereas the `busybox --list` output is obviously usable.

Busybox literally said --list was a thing I asked him to add in the commit
adding it, and that was after I explained it to them _twice_ on their own list
(which was ignored until I met him in person at ELC):

  http://lists.busybox.net/pipermail/busybox/2010-January/071229.html
  http://lists.busybox.net/pipermail/busybox/2010-March/071783.html

This is the problem with documentation. I say things, over and over, in many
different contexts until I'm SURE everybody's sick of me saying it, and then
even senior people go "I didn't know that".

(The problem is I've been there every time I said it, but each time maybe 2% of
my potential audience heard it if I'm _lucky_, and it's not even a _fresh_ 2%.
Marketing's got the "rule of 7": until somebody's been exposed to something 7
times they won't even remember they've ever seen it.)

And no, this is not even the first time for the meta-rant lamenting the rule of
7 is a thing:

  https://landley.net/notes-2017.html#24-12-2017
  https://landley.livejournal.com/41393.html
  https://www.openwall.com/lists/musl/2012/11/29/7

> (i never know with these things. on the one hand, i spend a lot of my
> time explaining "obvious unix stuff" to others, but then wasted a
> bunch of time this week paying the price for never having bothered to
> learn the difference between [ and [[ in the shell.

I'm learning all sorts of weird corner cases in the shell by writing a shell.
I've been quiet here because I'm mostly blogging it instead.

I learned SO much about how sed works by writing a sed, how mount works by
writing a mount... It is entirely possible I'm now one of the world's leading
experts on the Linux command line, which is less useful than you'd think.

> it's like the old
> thing about everyone uses 1% of MS Word, they just all use a different
> 1%...

That principle is one of my oldest objections to both C++ and Perl.

  https://landley.net/notes-2009.html#04-09-2008

I really liked python back when "there should be one obvious way to do it" was
still the language's motto. The Python 3 relaunch kinda fundamentally undermined
that, if you ask me...

> i have no idea here whether the current output fools other
> people into thinking they need to use tr or whatever before they can
> use it, but until you said a while back that IFS takes care of this it
> hadn't occurred to me.)

I never use IFS! But now I need to _implement_ IFS, so I gotta understand it.
(Which means sitting down, staring at the man page, and then writing a dozen
tests for "what does _this_ corner case do"...

>>>> Rob
>>
>> Still Rob

Getting back to "there should be a documentation section on this", the problem
is writing documentation (like writing tests) is time/energy consuming, and I
haven't even really got enough time to get this shell working in a reasonable
timeframe.

I wanted to have something promotable this release, but my current plan is to
fly back home on the 11th so probably release later this month, and I'm not
comfortable promoting what I've got yet. I'm working on subshell support now,
still need to do the other 80% of variable expansion, and then there's a whole
suite of builtins like export/set/readonly/declare... Plus variables have 10
categories ("help declare", see "options that set attributes" for the first 9,
the 10th is "magic" for things like $SECONDS and $RANDOM that basically call a
function unless  they've been unset since shell start)...

Darn it, when I'm doing the nommu data marshalling to a subshell through the
pipe (because vfork needs an exec to unblock the parent so globals go bye-bye) I
need to transmit which magics have been UNSET, because those variables _not_
being there is important. (Well, is it?)

  $ unset SECONDS; cat <(echo $SECONDS)

Yes it is, they don't respawn in subshells. And yes, it's unset: assignments are
ignored until a magic variable is unset:

  $ bash -c 'SECONDS=walrus; echo $SECONDS'
  0

Which differs from readonly:

  $ bash -c 'UID=47; echo $?'
  bash: UID: readonly variable
  1

Anyway, I've got local, global, readonly, and the first half of magic. But not
integer, the upper/lower case convversion, or references (you can symlink
variable names!).

I have yet to open the can of worms that is array variables, which appears to
have multiple types (indexed and associative: arrays and dictionaries, whee;
gotta finish lib/arbys.c and check it in at some point, I wanna implement hash
stuff with red/black trees because there are no fixed hash bucket sizes without
a pathological usage pattern.)

And I'm probably _not_ doing declare -t (trace). Unless somebody really really
wants it.

But I get today to bang on it, and tomorrow I'm back on $DAYJOB stuff. It would
be nice if I could get "sh -c '(echo hello)'" to stop segfaulting, but that's a
"the ( logic parses but isn't implemented at runtime" problem. Working on it,
this is a different category of input data than <(blah) was because that's a
quoted string that hasn't been parsed yet and this is already broken down into
an array with block annotation by the time it needs to spawn a subshell, but you
can't do a retroactive fork() on nommu and marshalling through the pipe up front
means turning it _back_ into a string...

*shrug* The usual...

Rob

P.S. Grumble grumble sh -c 'cat <(echo hello)' works on both nommu and with-mmu
but it's exiting before the command completes so the prompt of the host shell
prints before the hello does. Grrr. Looks like the wait logic is getting
confused by there being _two_ children, but it's a waitpid()! Grrr... Throw it
on the todo heap. Hmmm, how do I make a _test_ for that?