[Toybox] Would someone please explain what bash is doing here?

[Rob Landley]

On 5/20/20 8:39 AM, Chet Ramey wrote:
> On 5/19/20 4:03 PM, Rob Landley wrote:
>> On 5/18/20 10:41 AM, Chet Ramey wrote:
>>> On 5/17/20 7:11 AM, Rob Landley wrote:
>>>> I had a reply window open to this when my laptop battery died, and thunderbird
>>>> doesn't store unfinished messages like kmail and vi and chrome...
>>>>
>>>> Anyway, I was reminded of this thread by:
>>>>
>>>>   $ IFS=x; ABC=cxd; for i in +($ABC); do echo =$i=; done
>>>>   =+(c=
>>>>   =d)=
>>>>   $ bash -c 'IFS=x; ABC=cxd; for i in +($ABC); do echo =$i=; done'
>>>>   bash: -c: line 0: syntax error near unexpected token `('
>>>>   bash: -c: line 0: `IFS=x; ABC=cxd; for i in +($ABC); do echo =$i=; done'
>>>>   $ readlink -f /proc/$$/exe
>>>>   /bin/bash
>>>
>>> Yes, you need extglob to get a word like +(xyz) to parse unquoted. Since
>>> the word list following `in' is subject to pathname expansion, it's valid
>>> in that context.
>>
>> But the same command line in the current shell and in bash -c are parsing
>> different despite presumably reading the same .bashrc and friends?
> 
> My guess is that the interactive shell has extglob enabled somehow, whether
> it was in .bashrc or interactively after it was started. The -c command
> shell doesn't read startup files, so that doesn't matter.

Mine too, but grep isn't finding extglob under /etc or ~/.*

>> $ bash -c 'shopt -s extglob; IFS=x; ABC=cxd; for i in +($ABC); do echo =$i=; done'
>> bash: -c: line 0: syntax error near unexpected token `('
>> bash: -c: line 0: `shopt -s extglob; IFS=x; ABC=cxd; for i in +($ABC); do echo
>> =$i=; done'
>>
>> Nope, that wasn't it either.
> 
> This is the same thing that keeps aliases from being defined and expanded
> in the same compound command.

... they're not?

Why not?

> Enabling extglob changes the behavior of the
> shell parser. Since the shell always reads at least one line of input, and
> parses all of the commands on that line before executing any of them, the
> shopt will not be executed before the shell reads +($ABC).


  $ bash -c $'shopt -s extglob\nIFS=x; ABC=cxd; for i in +($ABC); do echo =$i=;
done'
  =+(c=
  =d)=

Sigh. Ok, fine. I should add a test for that.

  $ bash -ic $'IFS=x; ABC=cxd; for i in +($ABC); do echo =$i=; done'
  =+(c=
  =d)=
  $ strace bash -i -c 'exit' 2>&1 | grep open | wc -l
  45

Is it one of those files, or did debian hack their bash source to change the
builtin behavior? (Not going down this rathole right now.)

>>> No. Kind of in the middle. Consider the following:
>>>
>>> echo \
>>> one \
>>> two \
>>> $LINENO \
>>> done
>>>
>>> Bash will always expand $LINENO to 2 in this construct, since it was on
>>> line 2 when it figured out it was parsing a simple command.
>>
>> *blink* *blink.
>>
>> Ok, that just makes NO sense to me.
> 
> This is 100% what we talked about last week. The parser has to look ahead
> to determine whether or not it's parsing a function definition, and by the
> time it figures that out (after it gets the `one' from the lexer), we're
> on the second line. That's the line number that gets attached to the one-
> line simple command.

  $ potato\
  > (\
  > )\
  > { echo hello; }

Hmmm...

  $ potato\
  > \
  > (
  bash: syntax error near unexpected token `newline'

You're right, it does. I'm not doing that.

So in bash, $LINENO isn't the starting line of a multi-line span, and it isn't
the ending line of a multi-line span, it's the SECOND line of a multi-line span.
I cannot come up with any way to explain that other than "implementation detail
bubbling to the surface", and being unable to _document_ something has always
made me reluctant to _implement_ it. Hmmm...

>>> You fell victim to one of the classic blunders.
>>
>> I have neither gotten involved in a land war in asia nor gone up against a
>> sicilian when death is on the line (although the second one appears negotiable).
> 
> The second is only slightly less well-known.

Well he was right that everyone would be wearing masks in the future, I'll give
him that.

>> Yeah, I wrote my own parsing plumbing:
>>
>>   https://github.com/landley/toybox/blob/master/lib/args.c#L74
>>
>> It gets called automatically before each COMMAND_main() function, so the options
>> are already parsed and the results waiting before the command-specific C logic
>> starts running.
> 
> That looks clever.

Thanks. I banged on busybox for 5 years before starting over with toybox, and
had the advantage of knowing what I wanted and designing to provide it. (Knowing
what success looks like is often the hard part.)

>> Google/alphabet/android itself has given me the Google Open Source award twice,
>> each time with a $200 payoneer gift card (I spent the first one on japanese
>> lessons and an online game, the second needs some website activation thing that
>> didn't work, I should try again someday). 
> 
> I got one of those. It was for Google merch, though. :-) I still have some
> of it.

I should really activate the second card. They degrade over time if you don't.
(Payoneer is one of those places that serves the Unbanked, I.E. it's like check
cashing places, payday loans, and dollar stores: designed to profit off the poor
and thus somewhere between usurious and outright predatory. It's not the worst
of that genre, but it's kinda baked into the business model.)

>> Beyond that... Elliott bought me lunch once, and Tim Bird of Sony paid my travel
>> and hotel to ELC one year (but I think that might have been back when I was
>> doing busybox, not toybox? 
> 
> The FSF bought me a computer to do development on after shellshock. It's
> not all bad.

Cool.

>> That's the only place it's freely published. There were stupid "pay me for
>> printouts" versions
> 
> There still are.

I reached the point of "if it's not available free on the web, it's not a
standard" something like 15 years ago. (Which means I don't care about ACM
anything, I'm not a member and they can go hang.)

>> That's the one they set up when they got kicked off of comp.os.minix when
>> Tanenbaum came back for the new semester and started objecting to the traffic:
>>
>>   https://www.oreilly.com/openbook/opensources/book/appa.html
>>
>> Fascinating stuff, if you have a computer history hobby like I do. :)
> 
> Yep, I was there for a lot of that.

I was still in college.

I first encountered Linux when the 4 SLS floppy images came across Fidonet in
1993, went over to a friend's house who had a computer to install it on (xfree86
utterly refused to recognize his diamond stealth multimedia card and tried to
come up in 320x200 resolution), played with the text mode a bit (no dos box?),
went "why would anyone clone a sun workstation?" and wandered off again.

But I kept tabs on it because I knew about freeware (and used plenty like
binkleyterm and zmodem and pkzip) and knew that development couldn't be STOPPED
by commercial forces (you can't deprive volunteers of revenue). Plus the patch
program was AMAZING to somebody whose open source community was "if you register
wwiv you get the source code and the mode community exchanges emails explaining
how to upgrade it, and they look like this":

  https://landley.net/history/wwiv2/

I started following the Linux mailing list in something like 1996, and finally
got a working install in 1998 (and I had to buy a new computer with supported
hardware to do it because xfree86 never recognized ANY GRAPHICS CARD I ACTUALLY
OWNED except for the western digital one that had missing locking so the screen
distorted when anything was actually moving), and then my first experience with
the Debian mailing list drove me firmly into the arms of Red Hat for the next 7
years...

Sigh. I blogged about all this already, don't need to do it again:

  https://landley.net/notes-2018.html#17-07-2018

Anyway, the point is Linux was about 7 years old when I really jumped onboard. I
tend not to be an early adopter since I break everything anyway and trying a new
thing pretty much guarantees it won't work for me. :)

(When I was something like 12 I was told to use an experimental library catalog
terminal that was going to replace the card catalogs, and I spaced off the end
of the input field which put the terminal into an endless redraw loop they had
to reboot the mainframe to stop. I got in trouble for that.)

>>>> I only started using bash in 1998. :)
>>>
>>> And it was 10 years old at that time. Man, we've come a long way.
>>
>> I started toybox in 2006. I'm old...
> 
> We're both old.

Leaving posterity in a good position is sort of the point of the exercise, though:

  https://www.youtube.com/watch?v=5ZLtcTZP2js#t=3m

Rob