[Toybox] Testing: how to check return code (Rob Landley)

[scsijon]

> Message: 2
> Date: Sun, 3 Nov 2019 19:01:14 -0600
> From: Rob Landley <rob at landley.net>
> To: toybox at lists.landley.net
> Subject: Re: [Toybox] Testing: how to check return code
> Message-ID: <c3a291b2-bd33-0854-f5ef-9f97e1fe7a79 at landley.net>
> Content-Type: text/plain; charset=utf-8
> 
> On 10/30/19 1:56 PM, enh via Toybox wrote:
>>> I am with you enh. In this case, I wanted the error code because
>>> GNU `ls` says the command can exit with { 0, 1, 2 }.
>>> https://linux.die.net/man/1/ls
>>> POSIX says exit status is { 0, >0 }
>>> https://pubs.opengroup.org/onlinepubs/9699919799/
>>> In this case which specification wins?
>>
>> they're both true, but the GNU ls is more specific :-)
> 
> Hmmm, I need to add http://man7.org/linux/man-pages/dir_section_1.html to the
> roadmap, don't I? Except a quick cut and paste pipe through wc says there's just
> under 1500 entries in there...
> 
>>> enh, the other week you mentioned Android will not use the build if
>>> there are failing tests.
>>
>> correct. that's why i'm a bit behind --- xargs is still failing.
> 
> I thought I checked in your 4k workaround last week? (Commit f95d580892e2 ?)
> 
> Hmmm, just did a "git push" and it found stuff... (Sorry, Japan's been as busy
> as you'd expect.)
> 
>>> Does the above apply to `TEST_HOST=1
>>> make tests`? If so, then I will rewrite the test.
>>
>> not at the moment, no, but you probably want `SKIP_HOST=1` before the
>> affected `testing` (but on the same line so it doesn't apply to later
>> instances)?
> 
> I'm trying to migrate from SKIP_HOST to toyonly, because you could in theory
> have toybox on the host and the decision we're really making is "this is known
> to work in toybox, but not anywhere else".
> 
> Note: toyonly is a shell function that does an eval on its arguments instead of
> an environment variable that signals the plumbing to make a decision, so it's
> called slightly differently. But there should be a few examples in the tree...
> 
>> or fix ls to behave like the host. see grep for a similar example.
>> (or, generally, grep for `exitval`.)
> 
> I still wince at being too specific in the tests, but if there are actual
> examples of scripts caring about this level of detail and being broken by not
> getting it right... then the help text output should probably mention the error
> codes.
> 
> And _possibly_ we should work out some standard behavior to enforce across the
> commands? If "0" and "1" determine the result of a test (like cmp), neither of
> which is actually an error, then "2" being the default _error_ value everywhere
> might make sense? (But would we then break scripts expecting 1?)
> 
> I'm uncomfortable whack-a-moling this. I kinda want a rule.
> 
> Rob
> 
UM, I sort of wonder what we do with these?

 From http://www.tldp.org/LDP/abs/html/exitcodes.html, Table E-1. 
Reserved Exit Codes.

Especially the line below the table where they talk of exit codes 1 - 2, 
126 - 165, and 255, all having special meanings, and should therefore be 
avoided for user-specified exit parameters.