[Toybox] The cut -C test is failing because bionic's wcwidth() doesn't match glibc or musl.

[enh]

On Thu, Oct 21, 2021 at 12:18 AM Rob Landley <rob at landley.net> wrote:

> On 10/20/21 4:35 PM, enh wrote:
> > On Tue, Oct 19, 2021 at 11:56 PM Rob Landley <rob at landley.net
> > <mailto:rob at landley.net>> wrote:
> >     On 10/19/21 10:57 AM, enh wrote:
> >     > my guess is that you're using a statically-linked binary?
> >
> >     Yup. I can't test dynamic bionic on devuan, bionic isn't installed
> in /lib.
> >
> > if you don't mind creating a /system directory (or symlink) in /, you
> can just
> > run like that. there's a script in AOSP bionic that sets up all the
> links you
> > need to run the bionic tests on the host.
>
> [Removed flailing trying to find the script, you said where it is later.]
>
> Does it fix the OTHER warnings about being unable to "__bionic_open_tzdata:
> couldn't find any tzdata when looking for utc!" and such? I still need to
> fix
> the "FAIL: find -type f -user -exec" because bionic can't read my $USER
> out of
> /etc/passwd...
>
> Ahem:
>
> env: exec env: No such file or directory
> PASS: env -i
>
> Right, you disabled builtins,


yeah, to try to reduce mksh/toybox confusion. if/when we switch /bin/sh to
toybox it should reduce compat issues too.


> and there's no $PATH. I should $(which env) there.
> And maybe add an || echo fail
>
> (And now having done the $(which) annd checked it in... "env -i env" on
> the host
> does NOT have this failure. The -i does not apply to env's own exec.
> Sequencing
> I need to fiddle with...)
>
> All the other failures seem to be the unicode plumbing.
>
> >     (And when I _tried_ to create a chroot as a normal user, bionic
> segfaulted
> >     before calling main() because /dev/null wasn't there. Did that ever
> get fixed?
> >     Hard to run an init linked against bionic when that's the case.
> >
> > have you tried? i'm pretty sure there's a special case for pid 1, for
> exactly
> > this reason :-)
>
> I'm going to give you a moment to think about what you did.
>

yeah, it's one of those things that's hard to reason about --- "does my
tiger-repellant bracelet actually work, or are there just no tigers where i
live?". specifically "has Android never had a stdio confusion CVE because
of this, or because the world had already moved on from those security bugs
[which were all the rage when Android started], or just because we don't
have anything like inetd?". hard to untangle those, and since security --
like law -- is an ass-covering business, no-one wants to be the one to
_remove_ anything, even if it's not obvious that it's really helping. (and
in this case it's not obvious that it's _not_ helping, so...)

/me wonders whether i could at least replace the special case with a
fallback that uses open() with O_TMPFILE?


> >     The kernel guys
> >     never merged my https://lkml.org/lkml/2017/9/13/651
> >     <https://lkml.org/lkml/2017/9/13/651> stuff. I gave 'em three
> >     strikes and moved on. I have a todo item to teach cpio to
> hallucinate /dev
> >     nodes...)
> >
> >     > bionic doesn't have a
> >     > "static libdl", so when it tries to dlopen() icu4c to handle an
> i18n question,
> >
> >     ... why would it need to do that?
> >
> > because the last thing the world needs is _multiple_ copies of all the
> i18n
> > data. icu is the source of truth for that on Android. (we've had bugs in
> the
> > past in this area that made us clean up our act.)
>
> It's the dlopen() that hurts. I'd ask why there's no libicu.a, but the
> r21d ndk
> doesn't seem to have a libicu.so either. I should grab a newer version...
>

icu4c in apps is ... complicated. but you're never getting a *static* icu4c
for the same "the icu data format and schema aren't stable" reasons i've
mentioned before.


> Static linking is a legitimate use case. Android does not particularly
> treat it
> as such, but I'm going to keep periodically tapping the sign.
>

it's a legitimate use case _if you're prepared to carry everything around
yourself_. and that is supported --- feel free to build your own icu4c and
ship your own 30MiB data file :-)

(we did finally get a stable supported subset of the icu4c API for apps
into the NDK for S aka 12, but that doesn't help your use case.)


> >     Not sure if I _should_, but I _can_. (It was nice to leave this to
> libc. Then it
> >     wasn't my problem to update it every time Microsoft wrote another
> check to the
> >     unicode committee. Both glibc and musl can do this when statically
> linked.
> >     Sigh.)
> >
> > i'll only send a patch so i can disable this again on Android even if
> you _do_
> > do it, so i don't think it's a great use of your limited time :-)
>
> Oh there'd have been a config symbol, but I'd mostly convinced myself it's
> not
> worth it even before Rich's email.
>
> (The utf8 parsing was something glibc/musl/bionic all did differently, it
> was
> tiny anyway, and has changed only once since it was invented. The unicode
> stuff
> glibc and musl currently seem to do right, it's tractable but not trivial,
> it
> changes with the weather, and I am literally directly complaining to the
> person
> who maintains the one that fails.)
>
> > hang on, let me look out the symlink script... here you go:
> >
> >
> https://android.googlesource.com/platform/bionic/+/master/build/run-on-host.sh
>
> I was hoping that after the aosp build there'd be a system directory that I
> could ln -s /system to and it would just work. Instead:
>
>   # source this script in bash
>
>   source ${ANDROID_BUILD_TOP}/build/envsetup.sh
>
> So we have to do the envsetup in order for $ANDROID_BUILD_TOP to be set,
> and
> then this script does it _again_. Right.
>

(fwiw, i think that's for the buildbots.)


> Before I run something that makes this many assumptions as root so it can
> make
> presumably persistent changes to my host system, I think I'm going to try
> to
> read it a little more closely. Thanks for the pointer.
>

oh, yeah, to be clear: i was suggesting you boil this down to the three or
four lines you actually need (since aiui part of your goal is to _not_ have
to have a complete AOSP checkout that's up to date and built).


> > that should set you up with everything you need (and both 32- and 64-bit
> too, if
> > you're chasing something 32-bit-only).
>
> It declares a shell function, and then doesn't call it. What it DOES do is
> eval
> a soong invocation, and says to read envsetup.sh (which is just under 2000
> lines
> long) to figure out what --dumpvars-mode does (that env and declare -p
> presumably don't).
>

yeah, you just need the `ln -s`s. if you follow the history far back
enough, it used to basically just be that, but things got a lot more
complicated to support all the different kinds of builds, and the multiple
supported build environments (bots versus humans in particular).

if you just manually do the `ln -s` for /system/bin and /system/lib64 (and
/system/lib if you care about 32-bit), you're most of the way there, and i
think the error messages you get when you run will point you at the other
bits (tz data?).


> Probably not tonight.
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20211025/da027f80/attachment-0001.htm>