[Toybox] The cut -C test is failing because bionic's wcwidth() doesn't match glibc or musl.

[enh]

On Tue, Oct 26, 2021 at 10:54 PM Rob Landley <rob at landley.net> wrote:

> On 10/26/21 7:35 PM, enh via Toybox wrote:
> >     >     Right, you disabled builtins,
> >     >
> >     > yeah, to try to reduce mksh/toybox confusion. if/when we switch
> /bin/sh to
> >     > toybox it should reduce compat issues too.
>
> I note that toysh has to call both NOFORK and MAYFORK commands as builtins
> anyway. The NOFORKs for obvious reasons (cd in a child process won't
> help), and
> MAYFORK because the behavior changes (kill taking a jobspec argument as
> well as
> a PID, for example).
>
> But the config option still forces all the rest into seperate processes...
>
> P.S. A couple MAYFORKS I need to outright cheat for, ala:
>
>   $ time echo | sleep 3
>
>   real  0m3.003s
>   user  0m0.001s
>   sys   0m0.004s
>
> That's gotta be parsed _before_ the rest of the pipeline, which technically
> makes "time" it a flow control statement. Meanwhile, in bash:
>
>   $ echo | time sleep 3
>   bash: time: command not found
>
> Because of course. (Mine should work called both ways. It's on the todo
> list...)
>
> >     Doing so uncovered a legitimate bug (which I've been too buried to
> address but
> >     it's on the todo list) and made me think I might need to run the
> test suite with
> >     every combination of CFG_TOYBOX_* toggled, which is exponentially
> increasing on
> >     the number of options. :(
> >
> > it's definitely useful to at least test the "all toybox" configuration,
> since
> > that's basically what we test on Android (modulo mksh as the shell). but
> i catch
> > anything like that within a week or so anyway :-)
>
> Your testing is an excellent crutch, and I'm trying not to rely on it. :)
>
> >     What stops code from losing track of filehandles and calling
> close(1) and then
> >     opening a new stdout a moment later by accident? I remember
> segfaulting one of
> >     the gnu tools feeding it "-" as an argument more than once (it was a
> "what
> >     happens if" test implementing the toybox version and the answer, as
> is so often
> >     the case, is "nobody in gnu-land ever thought of that or tried it").
> >
> > fdsan:
> https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md
>
> Static tools: meet the halting problem. Halting problem, static tools.
>

i see you TL;DRed ... fdsan (like all the other *san tools it's named
after, despite not actually being related) is a dynamic tool :-)


> >     2) Defers the open() until there's an actual problem. Yes it calls
> open more
> >     than once if it has more than one thing to do (instead of dup) but
> it's one call
> >     made in a loop, and doesn't happen at all in the common case.
> >
> > yeah, that sounds reasonable. done
> > in https://android-review.googlesource.com/c/platform/bionic/+/1869594
>
> Yay! Thank you.
>
> >     So utf8 won't ever work in static bionic the same way DNS queries
> won't work in
> >     static glibc (because that needs to dlopen libnss).
> >
> > fancy stuff like widths won't work, no.
>
> Alas width is one of the few things I care about when parsing unicode,
> because
> it's used for text fitting (ps/top display fields and command line
> editing), and
> "is this a combining character or not" is the way to count unicode
> characters.
> (One non-combining codepoint plus five combining codepoints is one
> character.
> I'm still facepalming over the combining characters coming AFTER the
> character
> they combine with so you have to read PAST the end and then ungetc_w() to
> accurately delineate. Is there some advantage I'm missing to doing it that
> way?)
>
> >     >     Static linking is a legitimate use case. Android does not
> particularly
> >     treat it
> >     >     as such, but I'm going to keep periodically tapping the sign.
> >     >
> >     > it's a legitimate use case _if you're prepared to carry everything
> around
> >     > yourself_. and that is supported --- feel free to build your own
> icu4c and
> >     ship
> >     > your own 30MiB data file :-)
> >
> >     *shrug* Musl seems to do it well enough for me. I admit I'm not a
> polyglot.
> >
> > yeah, the libc APIs aren't really useful for any serious i18n work
> anyway (look
> > at the amount of detail lost in the translation from the icu4c API to the
> > wcwidth() one, for example).
>
> I'm not attempting to render them. That's the terminal's problem. I'm just
> trying to divide them into chunks I have to either truncate or stick
> newlines
> into. (Which is a hard enough problem on its own...)
>
> Rich Fellker made an x11 unicode display terminal once upon a time:
>
>   http://git.musl-libc.org/cgit/uuterm/
>
> But I haven't even taught toybox to use the 24 bit color ANSI escapes yet.
> :)
>
> >     > yeah, you just need the `ln -s`s.
> >
> >     The ones under:
> >
> >         if [ ${TARGET_ARCH} = x86 -o ${TARGET_ARCH} = x86_64 ]; then
> >
> >     Which apparently don't have a corresponding arm version?
> >
> > yeah, i don't have an arm host that's faster than a phone anyway.
>
> Building AOSP on a raspberry pi 4b should be a feasible thing to do?
>
>
> https://www.raspberrypi.com/products/raspberry-pi-4-model-b/specifications/
>
> It's 4xSMP at 1.5 ghz, can have 8 gigs ram,


i mean, i love my rpi400, and it's probably the most fun i've had with a
computer since the 1980s, but you don't want to build anything significant
on it. it's very noticeably an order of magnitude slower than my 10 year
old personal x86 laptop.


> and storage shouldn't be an issue
> either (Walmart is currently claiming to have a 2TB usb3 stick on sale for
> $12,
> and amazon says a 1TB sd card is $30. Can't vouch for either, but I bought
> a
> 512G sd card at Target that works fine so far?)
>

and i'd strongly recommend against doing anything on an sd card --- the few
times i've thought i've killed my rpi400 have actually been asking it to do
something i/o intensive on an sd card. most of the time you don't notice
they're awful, but try something "medium heavy" like checking out the linux
kernel onto an sd card and, well, it isn't pretty...


> It would be a "run overnight" kinda thing, but I don't think I've ever
> personally owned a system that's _not_ true for...
>

i hope that's not actually true... even a several year old laptop can build
AOSP in a few _hours_. hell, i have a MacBook "Pro" that can build AOSP in
a few hours, and between their awful keyboards and their general slowness
those things have a special circle in hell reserved just for them!



> Rob
> _______________________________________________
> Toybox mailing list
> Toybox at lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20211027/dc2c5506/attachment-0001.htm>