[Toybox] Pending (klogd, crond).

[Rob Landley]

On 10/26/24 06:09, Philippe PITTOLI wrote:
> Le Wed, Oct 23, 2024 at 07:06:50PM -0500, Rob Landley a écrit :
>> But I never understood "crontab". (Nor why posix has crontab but not crond.)
>> We don't have "sshtab", we have ~/.ssh/files that the daemon parses as
>> necessary. Why the heck would crond be any different?
> 
> I agree and that's one of probably many inconsistencies in the norm.
> 
> Now that you basically rewrote the whole userland, what would you
> change in the Unix/Posix standard to make it more consistent or
> practical maybe?

It's the qwerty keyboard problem. "Compatible" beats "sucks" when you 
take the installed base of user habits into account.

I've already nibbled at the edges a bit with things like cut -DF (which 
coreutils was still sitting on last I checked, 
https://lists.gnu.org/archive/html/coreutils/2024-06/msg00017.html has 
been a while so I should probably ping them yet again...)

> I'll start: a single regex engine for everything. I don't want to
> remember the differences between grep, sed, awk, vi(m)…

https://xkcd.com/927/

> I guess a big part of Unix is just a bunch of software put together,

Um, yes?

I went over this in https://landley.net/toybox/roadmap.html which could 
use an update but what else is new.

There are a half-dozen "standards" I've been using for toybox:

1) Posix
2) Linux Standard Base
3) linux man-pages
4) IETF RFC's
5) Linux kernel documentation
6) What's in Debian/LFS/busybox

Posix is active in the way a millipede is active: many legs go back and 
forth very fast in a way that does not result in much forward motion. 
But they've got an active mailing list 
(https://www.mail-archive.com/austin-group-l@opengroup.org/maillist.html) 
and truly horrific bugzilla and weekly meetings they post minutes of, 
and they just put out a new release this year (for the first time in 15 
years). But they took away the "download" link so it's kind of annoying 
to compare versions (you have to screen scrape and remove formatting 
issues). That said, this is the only standard I plan to go through line 
by line to fill out the test suite and at least document variances from 
before 1.0. It _mostly_ fails by omission. (Albeit some of them are 
really stupid, like replacing tar+cpio with "pax" and being resoundingly 
ignored for 20 years. Or the 2024 release still having SCCS in it, which 
was a predecessor of RCS, which was a predecessor of CVS, which was a 
predecessor of SVN.)

The Linux Standard Base vanished into the Free Software Foundation's 
accretion disk and was declared dead by Debian a decade ago: 
https://lwn.net/Articles/658809/ but has some interesting historical 
information. They also host a bunch of things like ELF documentation on 
the same refspects.linuxfoundation.org site, which aren't complete 
either (missing a lot of target ABI stuff and you need the old "linkers 
and loaders" book and maybe the "teensy elf" page to make sense of it). 
The Filesystem Hierarchy Standard kind of went in there too, but crap 
like /run showed up since that isn't mentioned in that old thing...

The Linux man-pages project was fine when Michael Kerrisk was 
maintaining it, but he stepped down a couple years back replaced by 
somebody who didn't even bother to host a web copy of the pages (so 
everybody still uses Kerrisk's man7.org) and then went "oh no I can't" 
and said he needed money 
(https://lwn.net/ml/all/4d7tq6a7febsoru3wjium4ekttuw2ouocv6jstdkthnacmzr6x@f2zfbe5hs7h5/ 
and no it hasn't been _close_ to 4 years since Michael stepped down) so 
that's nice. Kerrisks' man pages operated under a definition of "fine" 
that goes the other way from posix: it includes way too much crap: 
https://man7.org/linux/man-pages/dir_section_1.html has piles of 
commands for flatpak, mysql, "stacked git", yum _and_ dpkg, and random 
debris like "sheet2pcp". On the bright side, it's good at "document not 
legislate". (It says what existing commands do, not what it thinks they 
SHOULD do.)

The IETF RFCs have an even worse noise problem: there are THOUSANDS of 
them, maybe a couple hundred of which are interesting. Mostly they 
define protocols not commands. (A compression format is more or less a 
protocol, just for store-and-forward data instead of an interactive 
session.)

The kernel Documentation directory is mostly about how to talk to the 
kernel, especially filesystem-based APIs. (They don't actually document 
syscalls here: that's the man-pages project.) They don't say a lot about 
what command line programs should do. That said, when the kernel build 
decides it needs a new command out of the $PATH, I kind of have to add 
it (or maintain a patch to remove it) because mkroot builds the kernel 
with just toybox and a toolchain. (Well IDEALLY, there's still 
PENDING="expr git tr bash sh gzip   awk bison flex make ar" in 
scripts/install.sh but if I ever get over this burnout I need to shovel 
that out.)

The set of commands available in Debian's default install is a good 
reference for "what's considered available by default". Same for the set 
of packages/commands Linux From Scratch installs, and what busybox has 
been browbeaten into adding by its userbase (which is vaguely related to 
Alpine Linux these days, I think?)

> sometimes somewhat overlapping (with often a fine rationale like
> awk and sed), with a few shared design patterns but loosely enforced,
> etc.  Nothing really feels fundamental as in a mathematical sense,
> every piece is useful in the context of how the designers wanted
> to use their computer at the time.
> 
> Maybe we can (should?) do better.

I'm mostly trying to build existing packages and provide tools an 
established userbase can consume.

> And I'm not undermining the importance of what you are doing, the
> same tools with a better codebase already is a massive effort towards
> more hackable, predictable and fixable systems.

In part it's countering 'trusting trust':

https://www.landley.net/history/mirror/unix/ken/trustingtrust.html

http://lists.landley.net/pipermail/toybox-landley.net/2020-July/011898.html

And in part it would be really nice if a single semester community 
college course could walk people through an entire operating system 
build. But unfortunately there are FOUR conceptual packages in such a 
minimal build, only two of which (toybox and musl-libc) are currently 
being addressed. You need to replace gcc/llvm with a tinycc equivalent 
(and my own https://landley.net/code/tinycc fork of that is long dead) 
and do a replacement kernel. (Which people occasionally DO but so far 
they've screwed it up. I was excited about 
https://github.com/vvaltchev/tilck ala 
https://www.youtube.com/watch?v=Ce1pMlZO_mI last year but the source 
build can't NOT install its own compiler to build it under, and when I 
poked the maintainer about that he responded:

 >> I tried just typing "make" and it went:
 >>
 >>    Makefile:37: *** Before building Tilck, you need to build the
 >>    toolchain by running ./scripts/build_toolchain.  Stop.
 >>
 >> I tried setting it to one of my existing toolchains, which broke
 >> with:
 >>
 >>    $ TCROOT=~/mcm/ccc/i686-linux-musl-cross make
 >>    ...
 >>    CMake Error at CMakeLists.txt:118 (message):
 >>      /home/landley/mcm/ccc/i686-linux-musl-cross/.gcc_tc_ver_i386
 >>      not found: run scripts/build_toolchain
 >
> Hello Rob.
> There's no simple way to build Tilck without first running its
> build_toolchain script. Please, don't assume that the script is going to
> build a whole GCC toolchain from scratch: it's simply going to download 
> a pre-built one (~60 MB from toolchains.bootlin.com), along with busybox 
> etc. Here, "toolchain" doesn't mean just the GCC toolchain. It actually 
> means all the packages that Tilck will use for compiling user space
> projects.

At which point I moved on to other things...

I just want an understandable and maintainable base people can build on 
top of. Nothing's stopping you from adding packages, so "here's a better 
combined shell/sed/awk" is a package you can write yourself...

Except a man named Larry Wall already did that, and called it perl. No 
really, he literally said in his 1999 Atlanta Linux Showcase talk that 
that's where perl came from. And fell into the IETF failure mode of 
including too much (a lot of it redundant) instead of the posix failure 
mode of leaving a hole whenever there's disagreement ("we haven't 
defined init or mount so you can't actually BOOT a system that doesn't 
go beyond posix, but now we can call Windows NT and OS/360 posix 
compliant, please make the check out to..." It was because of FIPS 151-2 
which even 1980's Apple did a Unix https://en.wikipedia.org/wiki/A/UX 
and if the Federal Information Procurement Standards were requiring 
posix compliance as a condition of selling stuff to the government then 
of COURSE the big vendors did regulatory capture on that standards body).

*shrug* Still better than the Linux Foundation adding Red Hat Package 
Manager to LSB because Red Hat paid to be a plutonium member. Did I 
mention they're a 501c6 not a 501c3? People trip over that distinction a 
LOT: https://mstdn.jp/@landley/113371645666302993

Rob