[Toybox] Testing: how to check return code

[Rob Landley]

On 10/30/19 1:56 PM, enh via Toybox wrote:
>> I am with you enh. In this case, I wanted the error code because
>> GNU `ls` says the command can exit with { 0, 1, 2 }.
>> https://linux.die.net/man/1/ls
>> POSIX says exit status is { 0, >0 }
>> https://pubs.opengroup.org/onlinepubs/9699919799/
>> In this case which specification wins?
> 
> they're both true, but the GNU ls is more specific :-)

Hmmm, I need to add http://man7.org/linux/man-pages/dir_section_1.html to the
roadmap, don't I? Except a quick cut and paste pipe through wc says there's just
under 1500 entries in there...

>> enh, the other week you mentioned Android will not use the build if
>> there are failing tests.
> 
> correct. that's why i'm a bit behind --- xargs is still failing.

I thought I checked in your 4k workaround last week? (Commit f95d580892e2 ?)

Hmmm, just did a "git push" and it found stuff... (Sorry, Japan's been as busy
as you'd expect.)

>> Does the above apply to `TEST_HOST=1
>> make tests`? If so, then I will rewrite the test.
> 
> not at the moment, no, but you probably want `SKIP_HOST=1` before the
> affected `testing` (but on the same line so it doesn't apply to later
> instances)?

I'm trying to migrate from SKIP_HOST to toyonly, because you could in theory
have toybox on the host and the decision we're really making is "this is known
to work in toybox, but not anywhere else".

Note: toyonly is a shell function that does an eval on its arguments instead of
an environment variable that signals the plumbing to make a decision, so it's
called slightly differently. But there should be a few examples in the tree...

> or fix ls to behave like the host. see grep for a similar example.
> (or, generally, grep for `exitval`.)

I still wince at being too specific in the tests, but if there are actual
examples of scripts caring about this level of detail and being broken by not
getting it right... then the help text output should probably mention the error
codes.

And _possibly_ we should work out some standard behavior to enforce across the
commands? If "0" and "1" determine the result of a test (like cmp), neither of
which is actually an error, then "2" being the default _error_ value everywhere
might make sense? (But would we then break scripts expecting 1?)

I'm uncomfortable whack-a-moling this. I kinda want a rule.

Rob