[Toybox] The cut -C test is failing because bionic's wcwidth() doesn't match glibc or musl.

[Rob Landley]

On 10/26/21 7:35 PM, enh via Toybox wrote:
>     >     Right, you disabled builtins,
>     >
>     > yeah, to try to reduce mksh/toybox confusion. if/when we switch /bin/sh to
>     > toybox it should reduce compat issues too.

I note that toysh has to call both NOFORK and MAYFORK commands as builtins
anyway. The NOFORKs for obvious reasons (cd in a child process won't help), and
MAYFORK because the behavior changes (kill taking a jobspec argument as well as
a PID, for example).

But the config option still forces all the rest into seperate processes...

P.S. A couple MAYFORKS I need to outright cheat for, ala:

  $ time echo | sleep 3

  real	0m3.003s
  user	0m0.001s
  sys	0m0.004s

That's gotta be parsed _before_ the rest of the pipeline, which technically
makes "time" it a flow control statement. Meanwhile, in bash:

  $ echo | time sleep 3
  bash: time: command not found

Because of course. (Mine should work called both ways. It's on the todo list...)

>     Doing so uncovered a legitimate bug (which I've been too buried to address but
>     it's on the todo list) and made me think I might need to run the test suite with
>     every combination of CFG_TOYBOX_* toggled, which is exponentially increasing on
>     the number of options. :(
> 
> it's definitely useful to at least test the "all toybox" configuration, since
> that's basically what we test on Android (modulo mksh as the shell). but i catch
> anything like that within a week or so anyway :-)

Your testing is an excellent crutch, and I'm trying not to rely on it. :)

>     What stops code from losing track of filehandles and calling close(1) and then
>     opening a new stdout a moment later by accident? I remember segfaulting one of
>     the gnu tools feeding it "-" as an argument more than once (it was a "what
>     happens if" test implementing the toybox version and the answer, as is so often
>     the case, is "nobody in gnu-land ever thought of that or tried it").
> 
> fdsan: https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md

Static tools: meet the halting problem. Halting problem, static tools.
>     2) Defers the open() until there's an actual problem. Yes it calls open more
>     than once if it has more than one thing to do (instead of dup) but it's one call
>     made in a loop, and doesn't happen at all in the common case.
> 
> yeah, that sounds reasonable. done
> in https://android-review.googlesource.com/c/platform/bionic/+/1869594

Yay! Thank you.

>     So utf8 won't ever work in static bionic the same way DNS queries won't work in
>     static glibc (because that needs to dlopen libnss).
> 
> fancy stuff like widths won't work, no.

Alas width is one of the few things I care about when parsing unicode, because
it's used for text fitting (ps/top display fields and command line editing), and
"is this a combining character or not" is the way to count unicode characters.
(One non-combining codepoint plus five combining codepoints is one character.
I'm still facepalming over the combining characters coming AFTER the character
they combine with so you have to read PAST the end and then ungetc_w() to
accurately delineate. Is there some advantage I'm missing to doing it that way?)

>     >     Static linking is a legitimate use case. Android does not particularly
>     treat it
>     >     as such, but I'm going to keep periodically tapping the sign.
>     >
>     > it's a legitimate use case _if you're prepared to carry everything around
>     > yourself_. and that is supported --- feel free to build your own icu4c and
>     ship
>     > your own 30MiB data file :-)
> 
>     *shrug* Musl seems to do it well enough for me. I admit I'm not a polyglot.
> 
> yeah, the libc APIs aren't really useful for any serious i18n work anyway (look
> at the amount of detail lost in the translation from the icu4c API to the
> wcwidth() one, for example).

I'm not attempting to render them. That's the terminal's problem. I'm just
trying to divide them into chunks I have to either truncate or stick newlines
into. (Which is a hard enough problem on its own...)

Rich Fellker made an x11 unicode display terminal once upon a time:

  http://git.musl-libc.org/cgit/uuterm/

But I haven't even taught toybox to use the 24 bit color ANSI escapes yet. :)

>     > yeah, you just need the `ln -s`s.
> 
>     The ones under:
> 
>         if [ ${TARGET_ARCH} = x86 -o ${TARGET_ARCH} = x86_64 ]; then
> 
>     Which apparently don't have a corresponding arm version?
> 
> yeah, i don't have an arm host that's faster than a phone anyway.

Building AOSP on a raspberry pi 4b should be a feasible thing to do?

  https://www.raspberrypi.com/products/raspberry-pi-4-model-b/specifications/

It's 4xSMP at 1.5 ghz, can have 8 gigs ram, and storage shouldn't be an issue
either (Walmart is currently claiming to have a 2TB usb3 stick on sale for $12,
and amazon says a 1TB sd card is $30. Can't vouch for either, but I bought a
512G sd card at Target that works fine so far?)

It would be a "run overnight" kinda thing, but I don't think I've ever
personally owned a system that's _not_ true for...

Rob