[Toybox] Would someone please explain what bash is doing here?

[Rob Landley]

On 5/6/20 2:32 PM, Chet Ramey wrote:
> On 5/6/20 2:08 PM, Rob Landley wrote:
> 
>>> You're blogging these bash corner cases, too?
>>
>> I was. I was recently asked to stop.
> 
> Who asked you to stop?

Somebody on patreon. (I also stopped patreoning.)

  https://www.patreon.com/posts/36828541

(I miss the days when patreon actually let you post html without trying to
half-assedly parse it. I consistently put <p> and <a href= and so on everywhere,
which has vanished in SOME places but not others. Oh well, not my problem anymore.)

>> If you change them, that makes bash a moving target. 
> 
> Bash is evolving, not dead. There is room for bug fixes and improvements
> everywhere. I'm not going to stop fixing things, even if they've been there
> for a long time, if I think the behavior is wrong. Conversely, if I don't
> think the behavior is wrong, backwards compatibility is an important
> consideration.

I'm just trying to figure out what the behavior _is_.

Each of these is run in a fresh tab:

  $ echo $LINENO
  1

  $ echo \
  > $LINENO
  2

  $ echo $LINENO \
  $LINENO
  1 1

  $ bash -c $'echo \\\n$LINENO'
  1

Although that last one is at least consistent once I remembered:

  $ bash -c $'echo $LINENO \\\n$LINENO'
  0 0

>> (P.S. If you think I'm being meticulous, you should see what the actual security
>> professionals I used to follow on twitter do to this stuff: @aloria and
>> @hacks4pancakes and @fox0x01 and @0xabad1dea and @malwareunicorn and @evacide
>> and so on. 
> 
> Yep, I read them too when they come into my feed, and sometimes when they
> don't.

I miss twitter, but not enough to give them my phone number. (Which isn't a
secret, it's on my resume. But then they can't NOT use for password reset, and
even after
https://www.theverge.com/2020/4/27/21238131/twitter-sms-notifications-disabled-jack-dorsey-hack
they only disabled it for HIM, they still won't let me back into my account
without a modal "enter your phone number" blocking page immediately after log
in, and I refuse on principle.)

>>>> I currently have no IDEA what "sh --help" should look like when I'm done, 
>>>
>>> I'm pretty sure bash --help complies with whatever GNU coding standards
>>> cover that option.
>>
>> Currently 2/3 of bash --help lists the longopts, one per line, without saying
>> what they do. So yeah, that sounds like the GNU coding standards.
> 
> Hey, it's supposed to be brief. I did consider how to handle the long
> options, since it's easy to do the one-letter options, and decided there
> wasn't really a better way to do it.

The toybox way of conveying the information in bash's existing --help would be
something like:

---

Usage: bash [-ilrsDabefhkmnptuvxBCHP] [-c COMMAND] [-O SHOPT] [SCRIPT_FILE] ...

Long options:
	--debug --debugger --dump-po-strings --dump-strings --help --init-file
	--login --noediting --noprofile --norc --posix --rcfile --restricted
	--verbose --version

For -O SHOPT list 'bash -c "help set"', for more information 'bash -c help'
or visit https://www.gnu.org/software/bash or run "man 1 bash".

---

Except you've got some parsing subtlety in there I don't, namely:

  $ bash -hc 'echo $0' --norc
  --norc

  $ bash -h --norc -c 'echo $0'
  bash: --: invalid option

My generic command line option parsing plumbing can be told to stop at the first
non-option argument (for xargs and friends), and I think you can also annotate
an argument with that property (although it finishes the current -abcdef block,
and any pending "tar cCzfv dir filename" in FLAGS_NODASH mode).

But toybox is consistent about option parsing: if "ls -l blah --color" gives you
colored output, and "rm dir -r" lets wildcard expansions turn into control
flags, then it's going to do all the commands the same way. There are explicit
exceptions to this: "--help" and "--version" are globally parsed and only
accepted as the first argument (which was a compromise), and you can
TOYFLAG_NOHELP so "toybox false --help" reliably ignores its sarguments (which
people cared about)...

But I haven't got plumbing to let "sh -a --longopt -b" differ from "sh --longopt
-a -b" because it doesn't make _sense_ to do that.

(Even things like "git" mostly fall under the "stop parsing after the first
nonoption argument" case, it just needs to run through the multiplexer again
with a different command table because the first argument is a subcommand. "ip"
would probably behave the same way if I bothered to look at it, but abandoning
ifconfig and route because you didn't bother to update them for new APIs is
silly and I'd rather updating them for the new apis before paying any attention
to "ip".)

And some of this is just never going to parse the same way:

  $ bash -cs 'echo $0'
  bash

Mine would take the "s" as the argument to -c (because that's how the majority
of commands work and I value being consistent), which doesn't work here:

  $ bash -c'echo $0'
  allexport      	off
  braceexpand    	on
  ...
  xtrace         	off
  bash: - : invalid option
  landley at driftwood:~/toybox/t

Anyway, that means a condensed gnu bash usage: line would need [LONGOPTS] to say
where it knows how to parse them, which puts it over 80 chars, which is always
awkward...

Oh, the reason I capitalize words like that so if I refer to them later in the
help text it's obvious it's the same one from the usage line, ala:

    usage: uudecode [-o OUTFILE] [INFILE]

    -o  Write to OUTFILE instead of filename in header

But again, you have to conform to the gnu style guidelines, which I thought
means you'd have a texinfo page instead of a man page?

*shrug* Anyway, this is the kind of stuff I go through when writing my help
text. Pascal's apology again: you CAN write a shorter letter when that's an
explicit goal you're willing to put a lot of time into.

(Also, "help set" isn't just your -O SHOPT list, it's the vast majority of the
command line arguments as well: short options to set can be provided on the bash
command line and [-cilrsD] are _exceptions_ to that. Also, I dunno why -O blah
is a seprate namespace from "bash --pipefail", but then I made mount accept "-O
opt" and "--opt" as synonyms for each other back in the busybox days so I'm
presumably biased here. And yes, "mount --remount,rw dir" worked last I checked.)

Don't mind me, I'm like this. It's why I used to blog.

Hmmm...

----------
Usage: sh [--LONG] [-ilrsD] [-abefhkmnptuvxBCHP] [-c CMD] [-O OPT] [SCRIPT] ...

-c	Run CMD then exit (with -s continue reading from stdin)
-i	Interactive (prompt for commands on stderr, default if no -c or SCRIPT)
-l	Login shell (read /etc/profile ~/.bash_profile ~/bash_login ~/.profile)
-r	Restricted shell (crippled a dozen ways for "security")
-s	Read stdin commands (no SCRIPT, arguments are positional parameters)
-D	Does not work. (bash -D gives a prompt that ignores all commands typed.)
-O	see sh -c 'help shopt'

--LONG options:
	--debug --debugger --dump-po-strings --dump-strings --help --init-file
	--login --noediting --noprofile --norc --posix --rcfile --restricted
	--verbose --version

For options [-a-P] see 'bash -c "help set"', for more information 'bash -c help'
or visit https://www.gnu.org/software/bash or run "man 1 bash".
----------

Do you really need to document --help in the --help text? The bash man page does
not include the string "--debug" (it has --debugger but not --debug), and half
the rest is just gratuitous synonyms for short opts: --login is -l, --verbose is
-v, --restricted is -r, --dump-strings is -D which again:

$ bash --dump-strings
bash-4.4$ help
bash-4.4$ echo hello
bash-4.4$ exit
bash-4.4$ break
bash-4.4$ stop
bash-4.4$ ^C
bash-4.4$ ^C
bash-4.4$ ^C
bash-4.4$

Ahem. I'll stop now.

> Chet

Rob

P.S. --posix isn't -p, that's "privileged" mode which is not the same as
restricted mode and I'm walking away from the keyboard for a bit now.

P.P.S. the man page has --init-file but the --help output doesn't.