[Toybox] Debugging of the mkpasswd.c ASAN error (It's glibc's fault)

[Oliver Webb]

On Monday, March 4th, 2024 at 17:58, Rob Landley <rob at landley.net> wrote:
> Eh, it's us triggering it. Presumably we did something if a zillion other people
> haven't seen it. That said, a null pointer dereference isn't an off by one error
> or "allocation isn't quite large enough because the buffer's 22 bytes long and
> they're traversing it 32 bits at a time" or some such. That's "the logic took a
> wrong turn somewhere".

Did some more testing:
$ echo 'char *crypt(char *, char *); int main(void) { crypt("a", "AA"); }' | gcc -xc -fsanitize=address - -o mkpasswd && ./mkpasswd
[ASAN error]

$ echo 'char *crypt(char *, char *); int main(void) { crypt("a", "AA"); }' | gcc -xc - -o mkpasswd && ./mkpasswd
/sbin/ld: /tmp/ccKw47oU.o: in function `main':
<stdin>:(.text+0x19): undefined reference to `crypt'
collect2: error: ld returned 1 exit status

$ echo 'char *crypt(char *, char *); int main(void) { crypt("a", "AA"); }' | gcc -xc -fsanitize=address -lcrypt - -o mkpasswd && ./mkpasswd
$

Wha...

Okay, so ASAN is doing _something_ that replaces the call to crypt with something else, and since we only do
-lcrypt "as-needed" it does... something. Which means that crypt isn't really being called.
This is a WEIRD bug, why is ASAN replacing the symbol for crypt so we don't have to -lcrypt to get it...

The answer is to declare -lcrypt (-Wl,--as-needed doesn't work). While somehow keeping compatibility with musl
(which doesn't split libcrypt and libc). More porability.sh stuff, We'd need a mechanism to detect a glibc build tho

-   Oliver Webb <aquahobbyist at proton.me>