[Toybox] hexdump tests.

[enh]

On Wed, Mar 27, 2024 at 6:22 AM Rob Landley <rob at landley.net> wrote:
>
> On 3/25/24 10:42, enh wrote:
> > On Sun, Mar 24, 2024 at 1:40 AM Rob Landley <rob at landley.net> wrote:
> >>
> >> On 3/22/24 15:02, enh wrote:
> >> >> > CANONICALIZE_SPACE_IF_RUNNING_HOST_VERSION=1? so we trust ourselves but no-one
> >> >> > else? :-)
> >> >>
> >> >> I _don't_ trust myself, and I'm not special. (That's policy.)
> >> >
> >> > yeah, but that's why i suggested
> >> > CANONICALIZE_SPACE_IF_RUNNING_HOST_VERSION --- that way we can say "we
> >> > can't make hard assertions about the _host's_ whitespace, but we can
> >> > still make hard assertions about _ours_". if we just canonicalize all
> >> > the whitespace all the time, we can't (say) ensure that columns line
> >> > up or whatever.
> >>
> >> Or we could just "NOSPACE=1 TEST_HOST=1 make tests" if that's the test we want
> >> to run...?
> >
> > it's not though. that's my point. there are several cases:
> >
> > 1. testing toybox --- we know what whitespace we're expecting to
> > produce, and want tests to protect against regressions.
> >
> > 2. testing host tools --- we _don't_ have control over what whitespace
> > the host produces.
> >   a) in some cases we manually mark individual tests to show "we don't
> > care about host whitespace for this test case".
> >   b) sometimes this applies to _all_ the tests for a toy.
> >
> > we're talking about case 2b here, which is currently the
> > least-well-supported variant.
>
> You can NOSPACE=1 in an individual tests/command.test and it should last until
> the end of the file? That's why scripts/test.sh does:
>
>   # Run command.test in a subshell
>   (. "$1"; cd "$TESTDIR"; echo "$FAILCOUNT" > continue)
>
> So the variables and functions and so on defined in one test don't leak into
> others. I spent like 3 commits getting that to work properly, the last of which
> was commit 07bbc1f61280 and mentions the previous 2.
>
> > i think we're talking at cross purposes because _i'm_ talking about
> > variables set _within the tests, by the tests themselves_ and you're
> > talking about variables set on the command-line, which i don't think
> > make any sense here, because we're talking about properties of the
> > individual tests/commands.
>
> There are three scopes:
>
> 1) Variables exported into all tests
>
> POTATO=1 make tests
>
> 2) Variables set for a single test:
>
> POTATO=1 testcmd "thingy" "-x woo" "expected\n" "file" "stdin"
>
> 3) Variables set for the current test file.
>
> [ -n "$TEST_HOST" ] && NOSPACE=1
>
> Which is just a normal assignment (or export) in a tests/file.test, they go away
> at the end of the current file (because of the above parenthetical subshell
> calling it), and which was the new thing I added in 2022.
>
> I remember my first attempt at this years ago ctrl-c didn't work reliably, but
> the fix to that was just a trap at the top of scripts/test.sh:
>
>   # Kill child processes when we exit
>   trap 'kill $(jobs -p) 2>/dev/null; exit 1' INT
>
> > (unless you really do want to say "there's absolutely nothing we can
> > do about host whitespace, so give up completely", which i think has
> > yet to be proven that it's _that_ bad. but there are commands where
> > having a test that says "this whitespace -- that toybox produces -- is
> > reasonable [but as long as the non-whitespace matches, and there's
> > _some_ whitespace everywhere we have whitespace, we'll accept any
> > whitespace from the host tool]".)
>
> I think per-command [ -n "$TEST_HOST" ] && NOSPACE=1 might be reasonable. I'd
> rather not blanket do it for all commands.

+1. that's what we'd done so far, and i think it's been working pretty
well. i thought you were saying you wanted something with file scope,
but if not, i think we're in violent agreement to just keep doing what
we've been doing :-)

> Rob