[Toybox] Impact of global struct size

[Rob Landley]

On 1/5/24 19:14, enh wrote:
>> I'm out of the habit of speaking at conferences (there was a pandemic), really I
>> should just get on a regular local schedule of Posting Crap Videos To Youtube.
>> NOT trying to polish them but just get them out regularly and then later string
>> together playlists of the less bad ones. (I can blather much
>> stream-of-consciousness! You think this is bad, you should meet me in person!
>> Elliott was subject to this at a lunch once, and I was NOT sleep deprived, and
>> on my best behavior for that.)
> 
> (fwiw, it's less overwhelming in person where you're actually
> interacting than it is finding the time to even read a 500-line email
> response, let alone reply :-) )

I used to teach community college courses to vent my enthusiasm, but I wandered
away for several years and when I looked back into it the bureaucratic
requirements had increased dramatically. (Not certification, just... paperwork.)

> keep reading the test to see a couple of extra special cases:
> https://android.googlesource.com/platform/bionic/+/main/tests/unistd_test.cpp#1128

I was reading the actual kernel code to see what the limits are, which are
enforced here:

https://android.googlesource.com/platform/bionic/+/main/tests/unistd_test.cpp#1128

Based on _STK_LIM from:

https://github.com/torvalds/linux/blob/v6.6/include/uapi/linux/resource.h#L63

And ARG_MAX at:

https://github.com/torvalds/linux/blob/v6.6/include/uapi/linux/limits.h#L8

So min is 131072 bytes and max is 6 megabytes.

> basically, if RLIMIT_STACK is too big, you're capped at 128KiB. more
> weirdly, if it's too _small_ you also get the maximum 128KiB.

You're seeing 128k for too _big_? That's weird. Sounds like a bug?

> (i don't think this affects your code, but the reason we touched this
> recently is that it's not "32 pages" as we claimed before --- it's
> actually 128KiB, which _happens to be_ 32 pages if your page size is
> 4KiB: https://android.googlesource.com/platform/bionic/+/2da31cf7b0c6071f83244eb0c89f95395a48cb37%5E%21/#F0
> )

They hardwired 131072 into the ARG_MAX #define due to hysterical raisins, so it
didn't move when page size does. The comment in exec.c gives pages as historical
motivation, but that's not what the code _does_.

*shrug* Arbitrary and historical.

>> Basically I want to know what struct is at the end of the stack (a sequenced
>> collection of structs and arrays are conceptually in an encapsulating struct),
>> and where does "1/4 stack size" _start_ measuring from. (From the actual end, or
>> does some of the data there "not count"? The debian xargs behavior implies it's
>> _just_ measuring the strings, but if so I could feed it an argv[] of a couple
>> million "" and blow the stack because each of those is 8 bytes of argv[] to
>> point at 1 byte of NUL terminator, and resticting _that_ to 1/4 the stack would
>> try to write off the end of it. I'm pretty sure somebody would have noticed by
>> now...)
> 
> wasn't our conclusion last time we talked about this "it's always
> likely to wobble a bit, so as long as we go with the most conservative
> assumption, that's what we want for this use case?".

I was informed of "xargs --show-limits", and if I have to implement that I would
like to do it _right_.

Also, my argument with Linus about this was years ago now and things seem to
have stabilized. Or at least git annotate fs/exec.c says the kernel hasn't
changed how it calculates this since... commit 655c16a8ce9c1 in 2019 was peeling
out the calculation into a separate function... commit c31dbb146dd4 was fixing a
race condition (fetch stack limit once at the start of exec into a variable, in
case it changes during processing)...

Looks like commit da029c11e6b1 in 2017 was the last thing to touch this? 6 years
ago. That changed the behavior on July 7, and I argued with Linus about it
November 3:

https://lkml.iu.edu/hypermail/linux/kernel/1711.0/02949.html

Both the min and max limits the kernel enforces are complete ass-pulls (and
"max" seems likely to move someday because find | args is already choked by it
and I recently bought a raspberry-pi-alike with 8 gigs RAM), but we're about 6
months from the 7 year time horizon: seems stable enough that I can change my
code if/when they change again...

Rob