[Toybox] bash continues to confuse me.

[Chet Ramey]

On 7/1/20 7:28 AM, Rob Landley wrote:

>> Then the `why' as `because that's a choice Bourne made in 1978' should
>> suffice, right?
> 
> I'm not really trying to figure out why the behavior was chosen, that seems
> arbitrary and historic with archaeological layers.
> 
> I'm trying to figure out the minimum set of rules to capture the necessary
> behavior, and whether outliers from those rules are important or coincidental
> and possibly ignorable.

It seems like a clear rule is that a word expansion in braces beginning
with `#' finds the close brace and uses everything between the `#' and the
close brace as a parameter name, and expands to its length. POSIX says
pretty much exactly that.

> 
> (I have recently been reminded by another shell's maintainer that I'm not smart
> enough to have a chance of ever actually doing this, but I've never let that
> stop me before.)

I probably know a couple of maintainers who would say that, but it's rude.

>>>> Yes, variable indirection is an exception.
>>>
>>> One exception, yes. In bash, ${!!} doesn't indirect $!,
>>
>> It's true. That's never going to be useful, so bash just doesn't implement
>> `!' as one of the special parameters for which indirection is valid. But
>> you're right, it's inconsistent to not just accept it and expand to nothing.
>>
>>  ${#!} doesn't print the
>>> length of $!,
>>
>> Sure it does.
> 
>   $ echo ${#!}
>   bash: !}: event not found
>   $ echo "${#!}"
>   bash: !}: event not found

Ah, history expansion. Well, turn it off, I suppose. I use scripts or `bash
-c' to test this stuff.

> The context sensitive parsing doesn't do it in this case, but does
> for ${!} and ${!@} which is why I thought it would.

Yeah, the history expansion code doesn't know very much shell syntax. It's
part of readline. It originally didn't know any at all.


>>> I _think_ if bash says "bad substitution" and mine instead Does The Thing, that
>>> can't introduce an incompatibility in existing scripts? I think?
>>
>> Correct. Unless some script is, for whatever bizarre reason, counting on
>> the error.
> 
> Counting on the error is unlikely. Counting on a _lack_ of error for something
> broken that never triggers: plausible.

Fragile and unreliable. I don't have a lot of sympathy.

> 
>>>> The debatable part is whether or not to short-circuit
>>>> when the variable transformation code sees that the value it's being
>>>> asked to transform is NULL, but that's what mksh does and one of the
>>>> things I picked up when I experimented with the feature.
>>>
>>> Does that produce a different result? Seems like it's just an optimization? (Do
>>> you have a test case demonstrating the difference?)
>>
>> It's whether or not to flag an unrecognized transformation operator as an
>> error or just short-circuit before checking that because the value to be
>> transformed is NULL.
> 
> Yeah, I've been wondering about that, which definitely _can_ break scripts.
> 
> But implementing it seems tricky: ${x;%} reliably errors whether or not x is
> set, ${x~#%} never does (I can't find what ~ is supposed to do here in the man
> page,

Of course the first one errors -- `x;' is not a valid parameter name (and
we won't even talk about the missing pattern after `%'). I don't think
there's any argument there.

The second case is clearly a bug. It should produce an error.


> At a certain point I'm going to have to try scripts that break, and get bug
> reports from people who expected something to work and are angry at me.
> (Apparently it is vitally important that I care about a bash reimplementation of
> readline, which somehow manages to be both implemented in bash and to have a
> makefile. I've put it on the todo list.)

Why? The bash version of readline and the standalone version of readline
are identical. Identical consisting of the same source.

> 
>>> Which still leaves me with stuff like:
>>>
>>>   xx() { echo ${!@@};}; xx a
>>>
>>> Which.. I have no idea what it's doing? (The downside of the error behavior
>>> being "resolve to empty string".)
>>>
>>>   $ xx() { echo ${!@@};}; xx a b c
>>>   bash: a b c: bad substitution
>>
>> Already explained in the message you quoted.

Look back to the first paragraph of this message. :-)


>>  Part of that whole:
>>>
>>>   $ a=a
>>>   $ echo ${!a}
>>>   a
>>>   $ declare -n a
>>>   $ echo $a
>>>   bash: warning: a: circular name reference
>>>   $ echo ${!a}
>>>   bash: warning: a: circular name reference
>>>
>>> thing.
>>
>> Yes, if you're using namerefs, the nameref semantics for ${!var} take
>> precedence. Part of the nameref compatibility thing. It's documented
>> that way. namerefs are uglier and less valuable than I anticipated when
>> I implemented them.
> 
> I have not yet implemented namerefs. (Or integer variables. Or arrays. I've done
> local, global, and read only so far. Namerefs and arrays have read-side logic
> (declare -aAn), I think the rest (declare -ilrux) is all assignment-side?)

Pretty much.

> (I might skip declare -t: debug feature, possibly out of scope...)

For functions, it's something the debugger uses. For variables, it's a user
feature (a way to `tag' variables if you want to).

>>>>  although it makes:
>>>>>
>>>>>   $ x=?; echo ${!x/0/q}
>>>>>   q
>>>>>   $ x=^; echo ${!x/0/q}
>>>>>   bash: ^: bad substitution
>>>>>
>>>>> a bit harder to get right since as I said, my code for recognizing $? is in
>>>>> expand_arg() and not in varlen() or getvar(). 
>>>>
>>>> There are valid parameters, and there are invalid variable parameters.
>>>
>>> The man page calls them "special" parameters.
>>
>> There are some special parameters that are valid in indirections, but `^'
>> is not a special parameter.
> 
> Sure, I was using it to cause an error.

OK. I just didn't understand the reference to special parameters.

> 
> (And I have to pass
> _in_ the length it's allowed to consume so ${12} and $12 behave differently...)

Yeah, that's kind of a pain.

>> Sure. That's why declare reports `?' as not found. It's not a variable.
> 
> Which is why I needed to factor out the second function, which DOES know about
> it. (Sometimes it's a variable, sometimes it isn't...)

Well, if you want to split hairs, it's never a variable. Sometimes it's a
parameter.


>>>   $ echo ${*-yetthisislive}
>>>   yetthisislive
>>
>> Defined by posix.
> 
> When I get to the end of the bash man page (well, loop and do a pass with no
> hits), I intend to do a pass over posix-2008 to see what I missed. (I did read
> the whole thing once upon a time, it's just been a while.)
> 
> Until then I wince at every mention of it because when the _only_ reason for
> something is "posix"... ("Yes but why?" "Posix!")

I mean, that's not the only reason it behaves the way it does. POSIX, for
the most part, did a decent job of codifying ksh88 and the SVR4 sh, and
that expansion has been in there since Bourne wrote his shell.

> So this isn't "because posix", this one I understand the rule for, it's an
> operator category, although the category is fuzzed a little by ${:+} which
> shares the "maybe :" logic but triggers in the else case of that test.

Yes, `+' is the opposite of `-'; I feel like Bourne was being clever when
he decided on that syntax.


> I have a great big todo item to make a math parser. I did an elaborate one in
> java years ago that handled triginometric functions and fractional
> exponentiation and such. But that was a long enough time ago I still thought
> <strike>digital watches</a> Java was a pretty neat idea. I remember there were
> two stacks and I learned why reverse polish notation exists. You'd compare
> precedence to see whether you push the operation and argument, or perform it now
> and possibly consume your way down the tree, which is why I had to bother the
> posix guys to put the precedence BACK when they broke it in their html rendering
> of the expr command years ago because when I sat down to try it there they'd
> broken the spec...

That sounds ... complicated.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet at case.edu    http://tiswww.cwru.edu/~chet/