[Toybox] [SC22WG14.32335] Statement expressions

[Rob Landley]

On 7/15/25 12:09, JeanHeyd Meneide wrote:
> On Tue, Jul 15, 2025 Rob Landley <rob at landley.net> wrote:
>> So even though FILE * always had a variable storing the amount of
>> remaining input data (it HAS to) the member had a different name on
>> glibc and on musl and on bsd/mac, with no standard accessor functions,
>> and neither side wanted to standardize this because each felt it was the
>> other's responsibility.
> 
>       I'm sorry to hear that; that's got to be incredibly annoying and
> frustrating.

Eh, I don't blame a standard for having large holes in it. Beats 
specifying stuff I am 100% going to ignore forever (like posix trying to 
make "pax" a thing: will not happen, unix uses tar, rpm and initramfs 
are built around cpio, zip is off in its own corner, and it doesn't seem 
to occur to them to treat squashfs and mkisofs and gene2fs and so on as 
archivers).

But "talking to a standards body almost never solves problems I'm 
having" means I have very little incentive to update past C89. It was 
uint32_t and friends that got me to move TO C99... then I stopped using 
them once I learned about LP64. (And again, an actual web copy of C99 vs 
having to buy Herbert Schildt's book to get C89 because the "official" 
one was way too expensive for teenage me.)

Posix-2008 had openat() and friends, which I was already using and it 
was nice it acknowledged reality. Minimizing the delta between "what the 
spec says" and "what I'm actually doing" was a good reason to update 
from SUSv3 to SUSv4. I still have a proper close reading of SUSv5 on the 
todo list. (When it came out and I set aside spoons to go over it, they 
didn't have the downloadable tarballs up yet. Since then there's no 
urgency...)

Which is why I've always said a good standard should document, not 
legislate. When it tells me what to do and I already have a big delta vs 
it, that's just another thing to document a deviance against.

But when it describes in detail what is and isn't guaranteed to be 
portably available (so I have to do less testing, and can complain 
louder at tools that deviate pointing at "an authority"), that I find 
useful.

Your mileage, as always, may be measured in kilometers instead.

> Unfortunately, my (personal) appetite for doing things
> with the C standard library is limited and I frequently write off
> using the C standard library and roll my own stuff. Occasionally, when
> I feel like it's impossible to get around it (.e.g, because it's
> something owned thoroughly by the C standard library or its
> implementations and not something I can just work around), I try to
> help (e.g., Thread Interfaces, Encoding Interfaces, etc.).

I'm busy arguing with 
https://lore.kernel.org/all/6739da7d-e555-407a-b5cb-e5681da71056@landley.net/ 
and https://www.openwall.com/lists/musl/2025/07/12/6 and 
http://lists.landley.net/pipermail/toybox-landley.net/2025-July/030768.html 
and need to test 
https://patchew.org/linux/20250623-nolibc-sh-v2-0-0f5b4b303025@weissschuh.net/ 
and am STILL waiting for a decision on 
https://lists.gnu.org/archive/html/coreutils/2022-01/msg00007.html and 
haven't pushed anything to https://github.com/j-core?tab=repositories in 
a _year_ (the engineers have been busy, I just need to do the editing 
for publication) and of course my last "quarterly" toybox release was 
January (and the one before that the previous April)...

>       I recognize there's a huge gap in how stream stuff can and should
> behave, and how much information should be provided, but I personally
> have limited myself from working on the <stdio.h> header because it's
> very, VERY convoluted (as you're discovering with the POSIX/C Standard
> split). For example, I've received frequent requests to do something
> about `int` being the return type for most writing and reading
> functions and how that makes it untenable for large file I/O. I
> recognize it's a problem, but I'm not expert enough to come up with a
> solution as I just ignore stuff or use platform-specific I/O
> interfaces.

I'm not expert enough either, but I need to make it work, so there's a 
lot of incompetent flailing and shoveling. (I have read SO much kernel 
code trying to figure out what insufficiently documented syscall du jour 
actually DOES. I don't care what the standards body says if it doesn't 
match reality.)

The end result is then usually something like 
http://lists.landley.net/pipermail/toybox-landley.net/2016-March/004790.html 
where if I need to poke somebody else they ignore it and it never goes 
upstream into things that aren't "my" code unless I sit on upstream 
for... let's see, the perl removal patches took 5 years and so much 
explaining... https://lkml.iu.edu/hypermail/linux/kernel/1302.3/02061.html

I still type way too fast (see pascal's apology for writing a long 
letter because he didn't have the spoons to write a short one) but 
seldom have the patience to engage like with deaf upstreams anymore. 
Most things I _could_ put in 
https://landley.net/bin/mkroot/0.8.12/linux-patches/ and such I just 
don't bother because it's just another thing for version skew to break, 
and when one does drop out and go upstream it's usually by coincidence.

The irony is I started the first GPL enforcement suits 20 years ago, and 
what I proved was nobody had anything we actually _wanted_. Exactly one 
line of code went into busybox as a result of that effort, and it was 
literally just:

https://git.busybox.net/busybox/commit/?id=eb84a42fdd1d

There's a reason I do 0bsd now. :)

>       Given how deeply you've reached into this, maybe the best way to
> approach the problem on your end is to actually write a paper for WG14
> to look at, and then also show it to POSIX (the Austin Group) as a
> "hey, I'm serious about fixing this, now's the time to get a word in
> so you can be part of the solution".

I go down extremely deep ratholes all the time. But other people are 
usually territorial about their projects and I'm not "one of us". I 
present a thing and a rationale, they ignore it, we move on.

You wouldn't think 
https://lkml.iu.edu/hypermail/linux/kernel/2005.1/09399.html needed 
convincing, and yet.

I mean I reposted 
https://lkml.iu.edu/hypermail/linux/kernel/2202.0/01505.html as 
https://lkml.iu.edu/2302.2/08170.html and the response literally was:

"Yes we COULD make the linux build try "cc" first and automatically use 
llvm or gcc seamlessly, but we decided to force people to explicitly say 
LLVM=1 to use "clang" instead of "gcc" because that's the decision we 
made, so I don't want this patch to genericize it."

So that's nice.

> That way rather than just talking
> about it, they have something to work with (which might lead to a
> better solution). Figuring out the specification for that would
> probably be annoying and difficult as a first-timer, but.

I may be a first timer _here_, but I've been engaging with random third 
party projects for far longer than is healthy.

I just don't expect good results from any project that's been around 
longer than about 15 years. They tend to get very set in their ways. 
(Eyes toybox's first commit in 2006 dubiously...)

> Well. I
> never intended to work on typeof or anything; I was supposed to just
> do Unicode functions and leave.

I didn't MEAN to become busybox maintainer, or spend a year as the 
linux-kernel Documentation maintainer, they just sort of happened on the 
way to other things. (And I have long since moved on from both.)

>       But here we are. :)

I'm busy.

http://www.youtube.com/watch?v=SGmtP5Lg_t0

https://landley.net/aboriginal/about.html

http://lists.landley.net/pipermail/toybox-landley.net/2020-July/011898.html

>> Mostly I just test which compilers have what and make do with the
>> reality at hand. It's a lot easier when you write off Windows as "never
>> to be supported", then you can rely on things like LP64. :)
>>
>> If there was a standard that let me remove giant horrible things like:
>>
>> https://github.com/landley/toybox/blob/master/lib/portability.h
>> https://github.com/landley/toybox/blob/master/lib/portability.c
>>
>> I'd be more interested, but this seems unlikely.
> 
>       At least one thing I can point to being better, for
> https://github.com/landley/toybox/blob/master/lib/portability.h#L136-L168
> 
> - Endian stuff (and a lot of bit utilities) are in a new header called
> <stdbit.h> in C23.
> - The byteswap stuff took a longer time to fix (mea culpa: it took me
> too long to fix), and so is only in C2y
> https://thephd.dev/c2y-hitting-the-ground-running#more-bit-utilities

Uh-huh.

https://landley.net/toybox/faq.html#support_horizon

So I can start using stdbit.h in 2030, and the other one 7 years after 
you publish it.

>       The vast majority of the rest of this stuff looks like timers
> (POSIX stuff)/timespec failures (some of that addressed in C23, but
> it's also on vendors to not mess around with people's code and provide
> a conforming standard) and I/O work.

I'm still waiting for the Android NDK to ship the --static-libasan 
compatibility spelling from last year, which was 
http://lists.landley.net/pipermail/toybox-landley.net/2024-July/030444.html 
but happened again just https://github.com/landley/toybox/issues/554 ago.

And during all that there was some standard posix header that bionic 
didn't define a required function in... crypt() was it, where the header 
is there so __has_include() doesn't help but there's no way to #ifdef 
for whether or not a prototype is present and I can't just re-prototype 
it because theirs says things like "const" and "restrict" that mine 
never says.

And of course I hit that trying to work around 
http://lists.landley.net/pipermail/toybox-landley.net/2025-June/030730.html 
with https://github.com/landley/toybox/commit/635f1e0391b0 and by all 
means tell me how standards bodies are going to fix my problems.

> I'd love to say I've got a good
> grasp of the File I/O issues but in my head a lot of this stuff is
> "wow this could really use some fundamental tweaks and a fresh coat of
> paint", so I resist the urge to bother.

Every FILE * buffering option is silly, we need the nagle algorithm for 
stdio.

http://www.landley.net/notes-2024.html#28-04-2024

I also sketched out a memory mapped I/O ring buffer page the kernel 
could provide so userspace could check for recent input bytes before 
making a read() syscall to either wait or batch acknowledge them, but 
other than my "don't overshoot" use case it doesn't come up much. And 
how often do people read lines of input, that's a truly obscure use case 
clearly...

> A lot of this stuff looks like
> POSIX could be a lot more intentional about it; though, for the Apple
> fails, they source some of their libc from BSDs, so it might make
> sense to go improving things there and maybe it'll flow upstream?

I used to bump heads with Joreg Schilling a lot:

https://landley.net/notes-2016.html#11-03-2016

And basically said I wouldn't come back to posix until he died, and held 
to that position for years:

http://www.landley.net/notes-2021.html#19-06-2021

...and then he did, and I feel guilty. I mean I still think there's a 
legacy of terrible decisions due to him:

http://www.landley.net/notes-2024.html#15-07-2024

But I dialed in to one of the thursday meetings after he died to listen 
quietly and see if I could help. (It was all arguing over bugs I was 
unfamiliar with in areas I didn't care about, and the actual issues 
where about phrasing not working out correct behavior, so no, but still...)

To be honest: I'm a middle aged white guy past his prime, who never 
accepted systemd (despite 
http://www.landley.net/notes-2015.html#03-06-2015 but I think 
http://www.landley.net/notes-2014.html#04-09-2014 and 
http://www.landley.net/notes-2014.html#23-04-2014 are still true) and 
who thinks spraying down C projects with rust is a bad idea (don't cross 
contexts mid-project, you want to write a new one go do that but don't 
break existing ones, and don't act like you are "owed" existing projects 
you can't be bothered to write a new implementation to replace)...

I fit every model of "obsolete old grognard who thinks the stuff he grew 
up on is the better than today, and pines for a return to a glorious 
largely imagined past". I'm somewhat aware of it and try to compensate, 
but even _I_ don't want to hear from me. Why would I try to "steer" 
anything, or push anything that doesn't seem like an obvious fix into 
other contexts?

Lower the voting age to 12 and get out of the way. (You can register to 
vote in Scotland at 14, that's probably why 
https://www.youtube.com/watch?v=S-FT3W-rXsA is so catchy.)

>> (Honestly I'm probably mostly just writing C89 with compiler extensions.
>> With LP64 I don't really need uint32_t and friends... I guess %p wasn't
>> in c89? I claimed C99 because I had online copies of the C99 spec and
>> _didn't_ have online copies of C89...)
> 
>       That's how it always is, really. It's usually "C89, but I need a
> few things".

A corollary to Moore's Law is that 50% of what you known about 
programming the hardware is obsolete every 18 months. My personal 
corollary to THAT was the big advantage of unix is it's mostly the same 
50% cycling out over and over, with the bedrock/reef of posix staying 
between each hide tide.

I'd seriously hoped that systemd would be one of those tides, but alas 
"upstart" was created for a reason (because 
https://wiki.ubuntu.com/DashAsBinSh didn't work but south african white 
men never admit a mistake so it's sadly still there) and Ubuntu has 
basically never made a good technical decision (unity, mir...) and the 
people who did Red Hat's version have been blasting the "katamari 
damacy" theme at userspace so they can charge higher consulting fees 
replacing infrastructure people understood with infrastructure they have 
to pay to get supported...

Grumble grumble. SMP implies parallel boot scripts and we got a qwerty 
keyboard hairball solution from megalomaniacs. Oh well.

> My hope (with __has_include, better bit utilities, better
> atomics, better timespec, better threads, and more) is that we end up
> making a C2y/C3a enticing enough for people to say "yeah, actually, I
> have better things to do than maintain portability hacks: upgrade your
> compiler and demand better from your vendors".

Threads started as a hack on Slowaris because their fork was so slow.

https://www.landley.net/history/mirror/linux/kissedagirl.html

Java then required threading because they REFUSED to wrap poll/select in 
the java standard library, so you couldn't NOT use it.

Linux then acquired nontrivial threading support when Netscape released 
its source code at the start of 1998 and dumped its java userbase onto 
Linux (resulting in the famous 212% increase in one year). Solaris 
screwed over blackdown so badly that those developers mostly moved away 
from Java over the years, but they kept thinking in terms of threads.

I did plenty of threading on OS/2 before moving to Linux, but when I 
need SMP I can fork multiple processes that communicate via pipes and 
shared memory. I debug enough SMP lock coverage issues in the kernel, I 
don't want to try to reproduce terrible asynchronous issues where your 
test case has to run for 3 days to tell you whether or not you've fixed 
it in userspace as well. Not without a compelling REASON to do so. (Such 
as "this is the codebase I'm working a contract to try to debug, they're 
paying me to care". It's amazing how many people who need to fly someone 
in to clean up their mess make extensive use threads. Coincidence I'm sure.)

> Delivering core
> usability improvements is the only real way I see people enjoying
> working on low-level stuff with C for a long time to come.

Growing up I had a button "C combines the flexibility of assembly 
language with the power of assembly language".

C is a portable assembly language. It's the minimal amount of 
abstraction necessary so porting from x86 to arm isn't a complete 
rewrite, and no more.

The C++ people HATE when we say that, because their marketing message 
was always "C++ contains the whole of C and is therefore just as a good 
a language, the way this mud pie contains and entire glass of water and 
is thus just as good a beverage: QED". Tey HATE that every release they 
go "Look how much simplicity we added! This codebase has TWICE the 
simplicity as last time, and you can add even MORE simplicity via the 
boost libraries!" and somehow they have to make another release after 
that because it's still not right.

While the C people are still basically doing C89 with a couple tweaks 
because the language isn't broken.

>       Hopefully, eventually, the small things you want make it into C
> before you get tired of it or -- perhaps worse -- give up on it
> getting any better.

I don't really NEED it to get better. ? : works for me just fine, and 
has for a quarter century, it would just be nice if a "document not 
legislate" standards body caught up with the group.

> Thanks for sharing,
> JeanHeyd

Rob