[Toybox] new test failure on android

[enh]

On Thu, Jan 9, 2025 at 4:33 PM Rob Landley <rob at landley.net> wrote:
>
> On 1/9/25 11:36, enh wrote:
> > -- test
> > test: too many arguments
> > FAIL: test =~
> > echo -ne '' | "/system/bin/test" abc =~ a.c && echo yes
> > --- expected 2025-01-09 17:00:49.136005931 +0000
> > +++ actual 2025-01-09 17:00:49.144005931 +0000
> > @@ -1 +0,0 @@
> > -yes
>
> Hmmm... That's the "toyonly meets NOHELP" plumbing change, commit
> 58500b802905 specifically the:
>
> --- a/scripts/runtest.sh
> +++ b/scripts/runtest.sh
> @@ -102,7 +102,9 @@ skipnot()
>   # Skip this test (rest of command line) when not running toybox.
>   toyonly()
>   {
> -  IS_TOYBOX="$("$C" --version 2>/dev/null)"
> +  [ -z "$TEST_HOST" ] && IS_TOYBOX=toybox
> +  : ${IS_TOYBOX:=$("$C" --version 2>/dev/null | grep -o toybox)} \
> +    ${IS_TOYBOX:="$(basename $(readlink -f "$C"))"}
>     # Ideally we'd just check for "toybox", but toybox sed lies to make
> autoconf
>     # happy, so we have at least two things to check for.
>
> and
>
> --- a/tests/test.test
> +++ b/tests/test.test
> @@ -135,11 +135,10 @@ testing "<2" 'test def \< abc || echo yes' "yes\n"
> "" ""
>   testing ">1" 'test abc \> def || echo yes' "yes\n" "" ""
>   testing ">2" 'test def \> abc && echo yes' "yes\n" "" ""
>
> -# toyonly doesn't work with TOYFLAG_NOHELP
>   # bash only has this for [[ ]] but extra tests to _exclude_ silly...
> -#toyonly testcmd "=~" 'abc =~ a.c && echo yes' "yes\n" "" ""
> -#toyonly testcmd "=~ fail" 'abc =~ d.c; echo $?' '1\n' "" ""
> -#toyonly testcmd "=~ zero length match" 'abc =~ "1*" && echo yes'
> 'yes\n' '' ''
> +toyonly testcmd "=~" 'abc =~ a.c && echo yes' "yes\n" "" ""
> +toyonly testcmd "=~ fail" 'abc =~ d.c; echo $?' '1\n' "" ""
> +toyonly testcmd "=~ zero length match" 'abc =~ "1*" && echo yes'
> 'yes\n' '' ''
>
> parts.
>
> It looks like your test plumbing is testing something OTHER than toybox
> "test" (possibly bash's builtin test?) when it thinks it's testing
> toybox (TEST_HOST is not set, and readlink -f $C doesn't point to a file
> named toybox).

https://android.googlesource.com/platform/external/toybox/+/main/run-tests-on-android.sh

has

export C=\"\$(which $toy)\"; \

and `adb shell which test` does get me /system/bin/test, which is a
symlink to toybox...

(we also do the realpath check and warn if we're not actually testing
toybox [and ignore test failures in that case].)

running the test manually doesn't seem to work anyway?

$ adb shell
cheetah:/ # /system/bin/test abc =~ a.c
test: too many arguments
2|cheetah:/ # /system/bin/toybox test abc =~ a.c
test: too many arguments
2|cheetah:/ # test abc =~ a.c
/system/bin/sh: test: =/: unexpected operator/operand
2|cheetah:/ #


> The shenanigans around $C and using "testcmd 'blah'" instead of "testing
> 'test blah'" are to force the test to use the toybox version instead of
> things like shell builtins. (Which is working here for me, and I need to
> convert the REST of tests/test.test to testcmd to make sure it's not
> testing the bash builtin for the rest of them...)
>
> Rob