[Toybox] bash continues to confuse me.

[Chet Ramey]

On 6/27/20 9:02 PM, Rob Landley wrote:

>>> It's the circumstances under which ${!abc***} falls back to processing the ***
>>> part that I'm trying to work out.
>>>
>>>   $ ABC=123; echo ${!ABC:1:2}
>>>   21
>>>
>>> I _think_ what's happening is when you do ${!ABC@} it specifically checks for
>>> "@}" after the variable name, and if so it lists prefixes. (And [@] does the
>>> array thingy, haven't checked if that needs the } yet.) But if it doesn't end
>>> with THAT SPECIFIC SEQUENCE, it does the !ABC substitution FIRST and takes
>>> whatever's left after the original variable name (using normal "$WALRUS-x" logic
>>> to figure out where it ended) and does further slicing to do on _that_ result.
>>
>> More or less. The ${!var@} expansion is a special case, no question.  What
>> it does is pretty much what I described in my previous message, which you
>> end up quoting below:
> 
> Which makes ${!@} a special case within a special case? Ends with @} but doesn't
> trigger prefix logic. 

There is no such thing as a null prefix here.

> But that's just a length check. And ${!^@} can search the
> list without ever finding a match. 

What?

In this case, `!' is the parameter, `^' is the operator, and `@' is the
pattern for case modification. It can't be otherwise; `^' is not a variable
or special parameter.

> Which doesn't explain:
> 
>   $ echo ${!1@}
>   bash: ${!1@}: bad substitution

What's that supposed to do? It's not prefix matching, since `1' is not a
valid variable initial character. It's not indirection plus parameter
transformation, since there's no transformation operator. It's not straight
variable indirection, since `1@' isn't a valid parameter name.

>>> (I understand the implementation reason. I don't understand the _design_ reason.
>>> As a language... why?)
>>
>> I don't really care about the design reason. Whatever happened did so more
>> than 40 years ago, and it's a waste of effort to worry about it.
> 
> Except I'm trying to figure out what behavior to implement, so I kind of have to
> understand "why"...

Then the `why' as `because that's a choice Bourne made in 1978' should
suffice, right? The stuff in the braces after the `#' is the entire
parameter for the length expansion. It's an expansion unto itself. That's
how it's always worked. I don't feel like `extending' it.


>> Yes, variable indirection is an exception.
> 
> One exception, yes. In bash, ${!!} doesn't indirect $!,

It's true. That's never going to be useful, so bash just doesn't implement
`!' as one of the special parameters for which indirection is valid. But
you're right, it's inconsistent to not just accept it and expand to nothing.

 ${#!} doesn't print the
> length of $!,

Sure it does. You just have to have $! defined to get something useful.
Before you create an asynchronous process, it doesn't exist. If you haven't
created any background processes, you just get 0 for the length.

> ${!@Q} doesn't quote the value of $!,

See above. As it turns out, `@' is, in fact, one of the special parameters
you can indirect, so the parameter is `!@' and the `Q' makes it a bad
substitution.


 and I still don't understand
> why:
> 
>   $ xx() { echo "${*@Q}";}; xx a b c d
>   'a' 'b' 'c' 'd'
>   $ xx() { echo "${@@Q}";}; xx a b c d
>   'a' 'b' 'c' 'd'
> 
> produce the same output instead of the first one saying 'a b c d'.

Again, you can either believe the answer or not. I'm not going to keep
repeating it.

> 
> I _think_ if bash says "bad substitution" and mine instead Does The Thing, that
> can't introduce an incompatibility in existing scripts? I think?

Correct. Unless some script is, for whatever bizarre reason, counting on
the error.

> 
>> The debatable part is whether or not to short-circuit
>> when the variable transformation code sees that the value it's being
>> asked to transform is NULL, but that's what mksh does and one of the
>> things I picked up when I experimented with the feature.
> 
> Does that produce a different result? Seems like it's just an optimization? (Do
> you have a test case demonstrating the difference?)

It's whether or not to flag an unrecognized transformation operator as an
error or just short-circuit before checking that because the value to be
transformed is NULL.


> Which still leaves me with stuff like:
> 
>   xx() { echo ${!@@};}; xx a
> 
> Which.. I have no idea what it's doing? (The downside of the error behavior
> being "resolve to empty string".)
> 
>   $ xx() { echo ${!@@};}; xx a b c
>   bash: a b c: bad substitution

Already explained in the message you quoted.

>   $ xx() { echo ${!@@};}; xx a

So we get `a' as the variable passed to the transformation code. There are
a couple of different scenarios there. If there is no variable `a', you get
a NULL value, and the transformation code short-circuits. If there is a
variable named `a', it checks the transformation operator, which is invalid
and produces an error.

> 
>   $ xx() { echo ${!@@Q};}; xx a
> 

Same thing. If there were a variable `a', this would have quoted its value.
You have an example of that below.

>   $ xx() { echo ${!@Q};}; xx a
>   bash: ${!@Q}: bad substitution

Already explained.

>   $ xx() { echo ${!*Q};}; xx a
>   bash: ${!*Q}: bad substitution

This just doesn't mean anything. It's not even ambiguous.

>   $ xx() { echo ${!*@Q};}; xx a
> 

Still the same things with a variable named `a'.


>   $ xx() { echo ${!*+};}; xx a
> 
>   $ a=PATH; xx() { echo ${!@@Q};}; xx a
>   'PATH'

Yep, still the same things with a variable named `a'.

> Not having played with or read the code of 6 other shells, I am sadly at a loss
> here. I'm just trying to figure out a consistent set of rules for how bash
> behaves. (Or what the minimum number of rules to cover the maximum amount of
> bash behavior would be.)

Maybe we can make bash more consistent as a result, where that makes sense.

> 
>> On the other hand, if my code handles cases yours errors on,
>>> I'm not exactly being incompatible, am I?
>>>
>>> I mean I'm assuming:
>>>
>>>   $ ABC=def:1
>>>   $ def=12345
>>>   $ echo ${!ABC}
>>>   bash: def:1: bad substitution
>>>   $ echo ${def:1}
>>>   2345
>>>
>>> not working is maybe some sort of security thing,
>>
>> `def:1' is not a valid parameter name. In the second `echo', `def' is the
>> parameter name.
> 
> I was trying to determine order of operations on evaluation: the indirection
> results are not "live" for slice logic processing.

Indirection results are always a parameter name, not an operator.

 Part of that whole:
> 
>   $ a=a
>   $ echo ${!a}
>   a
>   $ declare -n a
>   $ echo $a
>   bash: warning: a: circular name reference
>   $ echo ${!a}
>   bash: warning: a: circular name reference
> 
> thing.

Yes, if you're using namerefs, the nameref semantics for ${!var} take
precedence. Part of the nameref compatibility thing. It's documented
that way. namerefs are uglier and less valuable than I anticipated when
I implemented them.

> 
>>  although it makes:
>>>
>>>   $ x=?; echo ${!x/0/q}
>>>   q
>>>   $ x=^; echo ${!x/0/q}
>>>   bash: ^: bad substitution
>>>
>>> a bit harder to get right since as I said, my code for recognizing $? is in
>>> expand_arg() and not in varlen() or getvar(). 
>>
>> There are valid parameters, and there are invalid variable parameters.
> 
> The man page calls them "special" parameters.

There are some special parameters that are valid in indirections, but `^'
is not a special parameter.

> 
>>> But it expand_arg seems to be
>>> where it belongs because there's a bunch of:
>>>
>>>   $ declare -p 'x'
>>>   declare -- x="^"
>>>   $ declare -p '?'
>>>   bash: declare: ?: not found
>>
>> Do you think declare should do syntax checking on whether or not its
>> arguments are valid identifiers? Even given that function names don't
>> have to be valid identifiers?
> 
> "declare -p" without parameters does not list $! and $? $0 and friends. They're
> a different category of variable.

Sure. That's why declare reports `?' as not found. It's not a variable.


> 
>   $ echo ${*@notanerror}

Well, you can short-circuit if there are no positional parameters (in which
case `*' ends up expanding to null), or you can error because neither `n'
(bash-5.0) nor `notanerror' is a valid transformation operator. It's the
same thing as above.

> 
>   $ echo ${*-yetthisislive}
>   yetthisislive

Defined by posix.

>   $ echo ${*:potato}
>   bash

`potato' is an arithmetic expression, which evaluates to 0, so it's the
same as echo ${*:0}, which expands to the positional parameters starting
from 0.


>   $ echo ${*=abc}
>   bash: $*: cannot assign in this way

Of course you can't. No shell lets you do this.

>>>   $ echo ${!walrus}
>>>   1
>>
>> So the variable indirection works.
> 
> My point was they're mostly doing the same thing, but aren't compatible.

Variable indirection and namerefs? They're not. Indirection was my attempt
to get most of the value out of namerefs without all of the internal
plumbing.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet at case.edu    http://tiswww.cwru.edu/~chet/