[Toybox] [PATCH] test.tar: fix tar tests on Android.

[Rob Landley]

On 7/9/19 8:23 AM, Gavin Howard wrote:
> Maybe I can help?

I come up for air maybe twice a day? You're not here? The coordination overhead
would swamp any possible productivity gain unless I could partition the task,
and then there's the "you could have written your own shell a year ago, why are
you asking me now" issue?

> My experience with bc's parser (and others) might help with
> parsing and tokenizing.

It's not how to parse, it's _what_ to parse. Making the code do what I want is
easy, figuring out exactly what it should do is what's taking all the time. For
example,last night I figured out that "{x}</dev/null echo $x" never producing
output is a sequencing issue, and

  $ if echo x1=$x; then echo x2=$x; fi {x}</dev/null
  x1=10
  x2=10

works fine. (And leaks filehandles! The way to close it is "exec 10<&-")

Also, { echo hello;}|cat needs both the space and the semicolon but (echo
hello)|cat doesn't, but _other_ than acting as a control character for
tokenizing, ) acts from a flow control perspective as a normal command
(specifically a flow control termination from an "(echo)echo" is an error but
"(echo)<<<abc" is not only ok, but applies retroactively to the start of the
statement, ala:

if echo hello
then
  echo walrus
fi | tee potato.txt

_both_ statements wind up in the file...

Oh, and the annoying part (in one of the twitter threads) is ";>&" and ";&>"
parse differently (the first is ";" ">&" and works after echo, the other is ";&"
">" and an error after echo), so you MUST consume specifically recognized groups
from the front when breaking it into words.

Oh, and "echo 2>/dev/null" produces no output but "echo 2|cat" produces output,
and the reason is _not_ the first glance one, it has to do with tokenizing.
("echo" "2>" "/dev/null" so echo gets called with no arguments, vs "echo" "2"
"|" "cat" so "echo 2" says 2\n which is then NOP washed through cat).

I've been flinging around shell script forever and I didn't know these things
because I just put spaces when it was ambiguous. Now I _need_ to know to run
other people's existing scripts, and there are SO many corner cases and they're
mostly historical accumulations with no obvious pattern behind most of them. And
no, posix isn't very helpful about tokenizing ";&" or "3<>" or ">|"...

And yes, "{x}<abc" parses as "{x}<" "abc", but:

  $ echo {abc
  {abc
  $ echo {abc</dev/null
  {abc

because of course.

I cut and paste these things to a big text file I need to turn into a much, much
longer tests/sh.test. My _code_ may not be getting longer fast, but that file is...

Rob