[Toybox] Config.probed changed, run 'make oldconfig'

[Rob Landley]

On 6/13/22 19:16, enh wrote:
> Sigh, this is another reason I use miniconfigs. These symbols would drop out in
> a miniconfig because they're constant within a toolchain. :P
>
>
> is this also why i need the "is not set" comment lines? i've never understood
> why i can't just miss out the irrelevant stuff...

Sadly, the "is not set" lines AREN'T comments. They LOOK like
comments, and anything ELSE starting with # is a comment, but the
menuconfig plumbing parses them because there are no "SYMBOL=n" lines
written out into the file, so when something needs to be explicitly
switched _off_, that line does it.

But that's not the bad part:

https://landley.net/notes-2014.html#13-08-2014

That stupidity SEEMS to have gone away recently, but trying to track
down what happened to it (and what they replaced it with) has hit the
weird github issue where "git bisect" has been sitting there for 20
minutes with one CPU pegged not producing a result yet. (Not the first
time I've triggered this behavior from git. Did I mention I break
everything? It usually resolves within 4 hours and then takes half as
long to do the next "bisect" step, and becomes tolerable a couple
cycles after that...

> The Config.probed symbols mostly control what _can_ be selected, hiding
> commands/options that aren't available in this build environment.
>
> But only mostly: TOYBOX_SHADOW specifically controls whether #include <shadow.h>
> happens in portability.h, that's a build break in _either_ direction if you get
> it wrong.
>
> That said, gcc 5 introduced __has_include so in theory I could use that if clang
> also supports it these days. (gcc 5.1 was April 2015, 7 year time horizon...)
>
> yes, clang's supported __has_include() for as long as we've been using clang.
> (annoyingly, their documentation really sucks -- far worse than gcc -- so i
> couldn't find a reference to what version it was introduced in.)

I bisect the source repository to answer this sort of question. (I am
fond of hammer/knife/fire style tools you can apply to a wide range of
issues. May not be RIGHT, but it WORKS...)

> Any compile time probes I can turn into build-time probes, I'm all for it.
>
> certainly __has_include(), yes, though see later for why syscalls are hairy...
>
> Figuring out how that impacts the dependency resolution is a design issue
> though. What's the right thing to do when this command depends on a feature
> that's not available in this build environment? Before menuconfig hid them, but
> now it wouldn't test it until compile time. I can probably figure out how to
> make commands drop out of the list (define HAS_BLAH() macros inside the
> __has_include stanzas maybe) and get --gc-sections to dead-code-eliminate
> them... but is that the right thing to do? Avoids a build break, at the cost of
> "I built this command and it's not in the resulting toybox binary". Is a build
> break better? A #warning output?
>
> (+1 to build break. your practice of "you don't have this constant from the
> future? here you go then..." [and then a possible runtime failure because you
> really are on too old a kernel] is also fine. but see later for my problem...)

> pointers. In uclibc fork() wasn't there on nommu targets, but musl provides a
> broken fork() that always returns -ENOSYS,
>
> this doesn't apply to fork(), but the trouble with ENOSYS on Android is that you
> have to ask yourself "was there a hole punched in the seccomp() filter for this
> syscall?". this is my concern with copy_file_range(), for example. although
> that's in really old kernels at this point, you can't call it on Android without
> getting SIGSYS courtesy of seccomp until <checks notes> next year in Android U.
> which is "fine" in a sense --- AOSP is "U" at this point, so this version of
> toybox is being built with a version of the OS that has a seccomp filter that
> will allow you to probe for copy_file_range(). but it's "bad" in that it means
> that you can't run even a static toybox binary from U on pre-U (at least not for
> a codepath that actually uses copy_file_range()).
>
> my initial plan (and the reason i don't think i even mentioned this when the
> recent copy_file_range() changes went in)

Which was an external contribution that I then fixed up, not something
I volunteered for. :P

> was to just keep it disabled for the
> next 7 years... but that's trickier if it's `#ifdef __NR_foo`ed like
> copy_file_range() now is, rather than a regular "config" item. obviously we can
> just add ` && !defined(__ANDROID__)` and a comment explaining why, but i'm
> tempted to wait until someone actually notices and complains?

It needs error handling falling back to the "didn't work, do the
read/write loop". And in THEORY it has that (try_cfr). In practice
it's awkward to test that this _works_...

> i'm _trying_ to
> persuade people to use a static toybox for hermetic testing on old android
> releases, but i'm not sure i've actually succeeded yet... ugh, no, i don't want
> to leave a known roadblock for those folks (_especially_ because i keep telling
> people to drive down that road!), so i'll send you a patch.

I agree with your use case and want it to work. I should figure out
how to have test coverage for this under mkroot...

> (before you complain, no, i wasn't a huge fan of the whole seccomp thing either,
> but if nothing else it's prevented kernels from shipping with "random" syscalls
> occupying the next few syscall numbers. which was a thing that happened: given a
> sufficiently large ecosystem, for anything you can imagine, someone's done it.)

The first implementation of the container stuff in OpenVZ was done
with new syscalls (multiple) instead of synthetic filesystems. Linus
did not choose to merge it...

> anyway, looking at the full list:
>
> * TOYBOX_CONTAINER seems years out of date, at least as a "does this compile?"
> probe? everyone should have setns()/unshare() by now?
>
> * TOYBOX_FIFREEZE seems years out of date? everyone should have that constant by
> now, and your usual way of doing that today would be to manually #define it anyway.
>
> * TOYBOX_UTMPX can be __has_include(<utmpx.h>)?
>
> * TOYBOX_SHADOW can be __has_include(<shadow.h>)?

I might have to reorder the #includes to implement that, I should look
at it when I'm not about to fall over...

> * TOYBOX_ANDROID_SCHEDPOLICY isn't used?

You removed its only user it commit b33d37d6f735 but left the symbol
in the probes.

> * TOYBOX_PRLIMIT is uclibc only? does uclibc have a __UCLIBC__ or something?

I've largely removed support for uClibc because it died. Musl replaced
it for all but a couple old nommu architectures. Buildroot beats a
uClibc-deadhorse fork in which I have zero interest. I'm following
glibc, musl, bionic, freebsd, netbsd, whatever macos is using, and
anything else should presumably adhere to posix.

> * TOYBOX_GETRANDOM can be __has_include(<sys/random.h>)?
>
> * TOYBOX_HASTIMERS is actually a workaround for a gcc issue, and should probably
> test the gcc version macros instead?

You mean the gcc version macros that clang _also_ defines, pretending
to be glibc? (And Intel's icc did that, and open64...) Sigh, if that
becomes the only remaining probed symbol I'll try to work something
out, but I don't trust them...

> i won't send a patch/patches in case you're already doing stuff in there, but
> let me know if you'd like any/all of those...

I've wandered onto a night schedule again and am about to fall over,
but I just got to a good checkpoint on the shell work and checked it
in, and will try to spend a few days on pending other stuff before
diving back in...

Rob

P.S. git managed to find the first revision, I tested it, did the "git
bisect good", and it's off eating CPU again...

P.P.S. Oh hey, I can't send this because thunderbird "upgraded" itself
and now instead of prompting me for the outgoing smtp server password
after a reboot it pops up an HTML window interacting with gmail in a
way that wants me to type in my email (um, how can it NOT already
know...? I was SURE this was phishing but it repeats?) and then when I
do it goes "there are two accounts for this email, an organization one
and a personal one, which would you like to use?" and selecting EITHER
of them goes back to the the "enter your email address" page in an
endless dysfunctional loop. I can still download mail, but not send.
Looks like I'm going to have to migrate my domain's mailserver to
dreamhost earlier than I expected. I'll probably have to log into the
web interface and cut and paste this message there...