[Toybox] [PATCH] Make it easier to switch regex implementations.

[Rob Landley]

On 11/2/20 1:55 PM, enh wrote:
> On Fri, Oct 30, 2020 at 7:12 PM Rob Landley <rob at landley.net> wrote:
>> On 10/28/20 7:06 PM, enh via Toybox wrote:
>>> One reason to use toybox on the host is to get the same behavior across
>>> Android/Linux/macOS. Unfortunately (as we've seen from a few bugs) one
>>> area where that doesn't quite work is that toybox uses the libc regular
>>> expression implementation.
>>
>> Which another reason the version of toybox you distribute should be statically
>> linked against bionic.
> 
> aye, but "host bionic" is a longer project with no-one funded to work on it.

What's actually involved? Mostly seems to be switching back to a minimal _start?

If there _is_ anything missing you care about (ls -l showing usernames?) then
license-wise, you should be able to pull anything you need from musl...

>>> That's fine, and mostly what users want, but
>>> those folks trying to get the exact same behavior everywhere might want
>>> to switch in a known regex implementation (bionic's NetBSD regex
>>> implementation, say) for increased consistency.
>>
>> By statically linking the binaries against bionic. :)
>>
>> (Did you ever fix the "hello world segfaults in a chroot that doesn't have
>> /dev/null because bionic's _start code does a lot with no error checking" issue?
> 
> no, that's actually a deliberate crash. that's definitely not a
> supported _device_ configuration, and we deliberately minimize the
> differences between host and device. (it's 99% of the point of having
> host bionic in the first place!)

Meaning you can't link PID 1 against bionic unless you have a static /dev, and
the kernel guys keep rejecting my "make initramfs honor CONFIG_DEVTMPFS_MOUNT"
patch. Sigh, I need to teach toybox cpio to accept non-filesystem metadata. It's
on the todo list...

>>> That actually works pretty well, but portability.h has an #ifndef test
>>> for REG_STARTEND before including <regex.h> that gets in the way. To
>>> make up for that, this patch removes the unnecessary #include <regex.h>
>>> from grep.c itself.
>>
>> Applied, but it's one measure of a whack-a-mole problem space.
> 
> there's never going to be a "host bionic for macOS" anyway, so this is
> necessary if not sufficient.

Statically linking on macos seems less of a minefield, though? They ship libtool
(which has nothing to do with gnu libtool, it's a combined ar+ranlib from next)
and it just works?

Glibc is uniquely broken because Ulrich Drepper personally hated static linking
(https://www.akkadia.org/drepper/no_static_linking.html) and intentionally
sabotaged it (moving things like name resolution behind dlopen), and when the
egcs guys... sorry, the eglibc guys were lured back into the fold and put
another committee in charge of the project instead of a maintainer (so nobody
would have authority to complain about the FSF (such as Drepper's polemic
against Stallman starting with "And now some not so nice things" in
https://sourceware.org/legacy-ml/libc-announce/2001/msg00000.html), the
committee was intentionally organized to be unable to tackle any remotely
political issues, and thus couldn't undo Drepper's anti-static prejudice without
admitting a mistake had been made by "the project". So they've been throwing
good code after bad ever since, and glibc is uniquely insane and broken as a
result. (Which means people have been inventing crap like "snap" and "flatpak"
so they can statically link dynamically linked programs by bundling up the
resulting filesystem and loopback mounting it in a container. Sigh. It's like
watching people build redstone computers in minecraft.)

Musl isn't broken that way, and neither is macos. I thought the point of host
bionic (other than behaving the same everywhere so catching bugs early and
avoiding potential distro skew and thus build dependencies for where the AOSP
prebuilts get updated from) is to avoid glibc's intentional sabotage of static
linking.

> unfortunately, something (presumably a kernel or file system change)
> seems to have broken a dd test on cuttlefish, so that's the next thing
> i need to look at when i can find some time:
> 
> FAIL: dd sync,noerror
> echo -ne "I WANT\n" > input
> echo -ne '' | dd if=input of=outFile seek=8860 bs=1M conv=sync,noerror
> 2>/dev/null &&
>    stat -c "%s" outFile && rm -f outFile
> --- expected 2020-10-29 16:10:58.647991948 -0500
> +++ actual 2020-10-29 16:10:58.667991947 -0500
> @@ -1 +1 @@
> -9291431936
> +701497344

I _really_ need to make a (5th?) attempt to clean up dd. But not until I can
carve out 2 uninterrupted weeks for it. Maybe new year's? I specced out the
block repacking layer it needs to insulate itself from that sort of thing
_years_ ago, and just... my todo list runneth over.

Rob