[Toybox] Config.probed changed, run 'make oldconfig'

[enh]

On Tue, Jun 14, 2022 at 6:13 AM Rob Landley <rob at landley.net> wrote:

> On 6/13/22 19:16, enh wrote:
> > Sigh, this is another reason I use miniconfigs. These symbols would drop
> out in
> > a miniconfig because they're constant within a toolchain. :P
> >
> >
> > is this also why i need the "is not set" comment lines? i've never
> understood
> > why i can't just miss out the irrelevant stuff...
>
> Sadly, the "is not set" lines AREN'T comments. They LOOK like
> comments, and anything ELSE starting with # is a comment, but the
> menuconfig plumbing parses them because there are no "SYMBOL=n" lines
> written out into the file, so when something needs to be explicitly
> switched _off_, that line does it.
>
> But that's not the bad part:
>
> https://landley.net/notes-2014.html#13-08-2014
>
> That stupidity SEEMS to have gone away recently, but trying to track
> down what happened to it (and what they replaced it with) has hit the
> weird github issue where "git bisect" has been sitting there for 20
> minutes with one CPU pegged not producing a result yet. (Not the first
> time I've triggered this behavior from git. Did I mention I break
> everything? It usually resolves within 4 hours and then takes half as
> long to do the next "bisect" step, and becomes tolerable a couple
> cycles after that...
>
> > The Config.probed symbols mostly control what _can_ be selected, hiding
> > commands/options that aren't available in this build environment.
> >
> > But only mostly: TOYBOX_SHADOW specifically controls whether #include
> <shadow.h>
> > happens in portability.h, that's a build break in _either_ direction if
> you get
> > it wrong.
> >
> > That said, gcc 5 introduced __has_include so in theory I could use that
> if clang
> > also supports it these days. (gcc 5.1 was April 2015, 7 year time
> horizon...)
> >
> > yes, clang's supported __has_include() for as long as we've been using
> clang.
> > (annoyingly, their documentation really sucks -- far worse than gcc --
> so i
> > couldn't find a reference to what version it was introduced in.)
>
> I bisect the source repository to answer this sort of question. (I am
> fond of hammer/knife/fire style tools you can apply to a wide range of
> issues. May not be RIGHT, but it WORKS...)
>
> > Any compile time probes I can turn into build-time probes, I'm all for
> it.
> >
> > certainly __has_include(), yes, though see later for why syscalls are
> hairy...
> >
> > Figuring out how that impacts the dependency resolution is a design issue
> > though. What's the right thing to do when this command depends on a
> feature
> > that's not available in this build environment? Before menuconfig hid
> them, but
> > now it wouldn't test it until compile time. I can probably figure out
> how to
> > make commands drop out of the list (define HAS_BLAH() macros inside the
> > __has_include stanzas maybe) and get --gc-sections to dead-code-eliminate
> > them... but is that the right thing to do? Avoids a build break, at the
> cost of
> > "I built this command and it's not in the resulting toybox binary". Is a
> build
> > break better? A #warning output?
> >
> > (+1 to build break. your practice of "you don't have this constant from
> the
> > future? here you go then..." [and then a possible runtime failure
> because you
> > really are on too old a kernel] is also fine. but see later for my
> problem...)
>
> > pointers. In uclibc fork() wasn't there on nommu targets, but musl
> provides a
> > broken fork() that always returns -ENOSYS,
> >
> > this doesn't apply to fork(), but the trouble with ENOSYS on Android is
> that you
> > have to ask yourself "was there a hole punched in the seccomp() filter
> for this
> > syscall?". this is my concern with copy_file_range(), for example.
> although
> > that's in really old kernels at this point, you can't call it on Android
> without
> > getting SIGSYS courtesy of seccomp until <checks notes> next year in
> Android U.
> > which is "fine" in a sense --- AOSP is "U" at this point, so this
> version of
> > toybox is being built with a version of the OS that has a seccomp filter
> that
> > will allow you to probe for copy_file_range(). but it's "bad" in that it
> means
> > that you can't run even a static toybox binary from U on pre-U (at least
> not for
> > a codepath that actually uses copy_file_range()).
> >
> > my initial plan (and the reason i don't think i even mentioned this when
> the
> > recent copy_file_range() changes went in)
>
> Which was an external contribution that I then fixed up, not something
> I volunteered for. :P
>
> > was to just keep it disabled for the
> > next 7 years... but that's trickier if it's `#ifdef __NR_foo`ed like
> > copy_file_range() now is, rather than a regular "config" item. obviously
> we can
> > just add ` && !defined(__ANDROID__)` and a comment explaining why, but
> i'm
> > tempted to wait until someone actually notices and complains?
>
> It needs error handling falling back to the "didn't work, do the
> read/write loop". And in THEORY it has that (try_cfr). In practice
> it's awkward to test that this _works_...
>

well, the `&& !defined(__ANDROID__)` should solve that for you --- we'll
always be testing the fallback path :-)

(and by the time we don't need that any more, you can probably remove the
fallback for everyone else too!)


> > i'm _trying_ to
> > persuade people to use a static toybox for hermetic testing on old
> android
> > releases, but i'm not sure i've actually succeeded yet... ugh, no, i
> don't want
> > to leave a known roadblock for those folks (_especially_ because i keep
> telling
> > people to drive down that road!), so i'll send you a patch.
>
> I agree with your use case and want it to work. I should figure out
> how to have test coverage for this under mkroot...
>

funnily enough (since, to my eternal annoyance, you can't configure-out
syscalls from a kernel) that's probably easiest done by going the other
route, catching SIGSYS, and installing a seccomp filter around a test.


> > (before you complain, no, i wasn't a huge fan of the whole seccomp thing
> either,
> > but if nothing else it's prevented kernels from shipping with "random"
> syscalls
> > occupying the next few syscall numbers. which was a thing that happened:
> given a
> > sufficiently large ecosystem, for anything you can imagine, someone's
> done it.)
>
> The first implementation of the container stuff in OpenVZ was done
> with new syscalls (multiple) instead of synthetic filesystems. Linus
> did not choose to merge it...
>
> > anyway, looking at the full list:
> >
> > * TOYBOX_CONTAINER seems years out of date, at least as a "does this
> compile?"
> > probe? everyone should have setns()/unshare() by now?
> >
> > * TOYBOX_FIFREEZE seems years out of date? everyone should have that
> constant by
> > now, and your usual way of doing that today would be to manually #define
> it anyway.
> >
> > * TOYBOX_UTMPX can be __has_include(<utmpx.h>)?
> >
> > * TOYBOX_SHADOW can be __has_include(<shadow.h>)?
>
> I might have to reorder the #includes to implement that, I should look
> at it when I'm not about to fall over...
>
> > * TOYBOX_ANDROID_SCHEDPOLICY isn't used?
>
> You removed its only user it commit b33d37d6f735 but left the symbol
> in the probes.
>

patch sent, since this one seems unobjectionable given that.


> > * TOYBOX_PRLIMIT is uclibc only? does uclibc have a __UCLIBC__ or
> something?
>
> I've largely removed support for uClibc because it died. Musl replaced
> it for all but a couple old nommu architectures. Buildroot beats a
> uClibc-deadhorse fork in which I have zero interest. I'm following
> glibc, musl, bionic, freebsd, netbsd, whatever macos is using, and
> anything else should presumably adhere to posix.
>
> > * TOYBOX_GETRANDOM can be __has_include(<sys/random.h>)?
> >
> > * TOYBOX_HASTIMERS is actually a workaround for a gcc issue, and should
> probably
> > test the gcc version macros instead?
>
> You mean the gcc version macros that clang _also_ defines, pretending
> to be glibc? (And Intel's icc did that, and open64...) Sigh, if that
> becomes the only remaining probed symbol I'll try to work something
> out, but I don't trust them...
>

yeah, that's annoying but you can always `!defined(__clang__) && `.


> > i won't send a patch/patches in case you're already doing stuff in
> there, but
> > let me know if you'd like any/all of those...
>
> I've wandered onto a night schedule again and am about to fall over,
> but I just got to a good checkpoint on the shell work and checked it
> in, and will try to spend a few days on pending other stuff before
> diving back in...
>
> Rob
>
> P.S. git managed to find the first revision, I tested it, did the "git
> bisect good", and it's off eating CPU again...
>
> P.P.S. Oh hey, I can't send this because thunderbird "upgraded" itself
> and now instead of prompting me for the outgoing smtp server password
> after a reboot it pops up an HTML window interacting with gmail in a
> way that wants me to type in my email (um, how can it NOT already
> know...? I was SURE this was phishing but it repeats?) and then when I
> do it goes "there are two accounts for this email, an organization one
> and a personal one, which would you like to use?" and selecting EITHER
> of them goes back to the the "enter your email address" page in an
> endless dysfunctional loop. I can still download mail, but not send.
> Looks like I'm going to have to migrate my domain's mailserver to
> dreamhost earlier than I expected. I'll probably have to log into the
> web interface and cut and paste this message there...
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20220614/513b72b9/attachment-0001.html>