[Toybox] hexdump tests.

[Rob Landley]

On 3/25/24 10:42, enh wrote:
> On Sun, Mar 24, 2024 at 1:40 AM Rob Landley <rob at landley.net> wrote:
>>
>> On 3/22/24 15:02, enh wrote:
>> >> > CANONICALIZE_SPACE_IF_RUNNING_HOST_VERSION=1? so we trust ourselves but no-one
>> >> > else? :-)
>> >>
>> >> I _don't_ trust myself, and I'm not special. (That's policy.)
>> >
>> > yeah, but that's why i suggested
>> > CANONICALIZE_SPACE_IF_RUNNING_HOST_VERSION --- that way we can say "we
>> > can't make hard assertions about the _host's_ whitespace, but we can
>> > still make hard assertions about _ours_". if we just canonicalize all
>> > the whitespace all the time, we can't (say) ensure that columns line
>> > up or whatever.
>>
>> Or we could just "NOSPACE=1 TEST_HOST=1 make tests" if that's the test we want
>> to run...?
> 
> it's not though. that's my point. there are several cases:
> 
> 1. testing toybox --- we know what whitespace we're expecting to
> produce, and want tests to protect against regressions.
> 
> 2. testing host tools --- we _don't_ have control over what whitespace
> the host produces.
>   a) in some cases we manually mark individual tests to show "we don't
> care about host whitespace for this test case".
>   b) sometimes this applies to _all_ the tests for a toy.
> 
> we're talking about case 2b here, which is currently the
> least-well-supported variant.

You can NOSPACE=1 in an individual tests/command.test and it should last until
the end of the file? That's why scripts/test.sh does:

  # Run command.test in a subshell
  (. "$1"; cd "$TESTDIR"; echo "$FAILCOUNT" > continue)

So the variables and functions and so on defined in one test don't leak into
others. I spent like 3 commits getting that to work properly, the last of which
was commit 07bbc1f61280 and mentions the previous 2.

> i think we're talking at cross purposes because _i'm_ talking about
> variables set _within the tests, by the tests themselves_ and you're
> talking about variables set on the command-line, which i don't think
> make any sense here, because we're talking about properties of the
> individual tests/commands.

There are three scopes:

1) Variables exported into all tests

POTATO=1 make tests

2) Variables set for a single test:

POTATO=1 testcmd "thingy" "-x woo" "expected\n" "file" "stdin"

3) Variables set for the current test file.

[ -n "$TEST_HOST" ] && NOSPACE=1

Which is just a normal assignment (or export) in a tests/file.test, they go away
at the end of the current file (because of the above parenthetical subshell
calling it), and which was the new thing I added in 2022.

I remember my first attempt at this years ago ctrl-c didn't work reliably, but
the fix to that was just a trap at the top of scripts/test.sh:

  # Kill child processes when we exit
  trap 'kill $(jobs -p) 2>/dev/null; exit 1' INT

> (unless you really do want to say "there's absolutely nothing we can
> do about host whitespace, so give up completely", which i think has
> yet to be proven that it's _that_ bad. but there are commands where
> having a test that says "this whitespace -- that toybox produces -- is
> reasonable [but as long as the non-whitespace matches, and there's
> _some_ whitespace everywhere we have whitespace, we'll accept any
> whitespace from the host tool]".)

I think per-command [ -n "$TEST_HOST" ] && NOSPACE=1 might be reasonable. I'd
rather not blanket do it for all commands.

Rob