[Toybox] Project progress for 0.8.3.

[enh]

On Sat, May 30, 2020 at 5:52 AM Rob Landley <rob at landley.net> wrote:
>
> On 5/29/20 2:00 PM, enh wrote:
> > On Sun, May 24, 2020 at 1:55 PM Rob Landley <rob at landley.net> wrote:
> >>
> >> Two weeks since the release and Wikipedia[citation needed] hasn't noticed yet,
> >> which is fine and normal (I just don't want them to be actively _wrong_), but
> >> their "Project progress" section explains that "in 2015"... which was 5 years
> >> ago now? And they still link to http://www.landley.net/toybox/todo.txt from 2011
> >> which is PURELY HISTORICAL...
> >
> > wikipedia doesn't update itself, you know :-)
> >
> > (if you were actually just trying to get someone to do it for you: done.)
>
> Editing wikipedia is against my religion.
>
> >> That leaves getopt, which is legitimately stuck in pending due to being a
> >> design-level nightmare: the one that's there works fine but pulls in a whole
> >> second set of option parsing logic from lib.c that nothing else uses.
> >
> > that's kind of the _point_ of that command :-)
>
> Maybe I can move it to toys/examples so it's not part of defconfig? No?

fine by me. i'm already using it from pending after all :-)

> Who _does_ use getopt (but not getopts)? I current need to implement getopts as
> part of the shell stuff, and can cycle back to looking at this after that.
> (Hopefully with enough information about what it either does that I could
> understand why anyone would want to use it.)

shell scripts that want to support long options. getopts only supports
short options.

(i've not seen anyone use getopt when they only have short options.)

> >> sha256sum and sha512sum are tangled into the "sha-3" todo item (see also
> >> sha224sum sha384sum and sha3sum). I did my own md5sum and sha1sum
> >> implementations way back when (which are merged and share code) and I want to
> >> see if I can fit the rest in there, but haven't sat down to really focus on it
> >> yet. (It's mathy, there's research. Like the compression algorithms. I do NOT
> >> wanna be distracted from chewing on it halfway through, so big chunka time.)
> >
> > for anyone happy with using libcrypto's optimized code, though, these
> > are all done. count them as 50%? :-)
>
> Oh sure. I think I covered that later.
>
> This isn't necessarily for "general use", this is for "auditable base" which
> must be self-contained with no external dependencies.
>
> Longer term I'd also like to do qcc and get the system building itself with that
> so the toolchain is _also_ part of the auditable base. The weakness in this
> theory is that I'd then use llvm's C backend to compile llvm to C and build that
> under qcc, to natively bootstrap up into an environment that can run insane C++
> specification du jour (no really, we've fixed it all THIS time, and if not try
> next year!)
>
> And the problem with building the C version of LLVM is how would you spot a
> trusting trust attack in LLVM? If it compiled itself to C code containing the
> exploit, which was faithfully compiled by qcc into a runnable binary (which you
> then rebuild with itself to get a 3x efficiency boost)... it's still a trusting
> trust exploit injection vector.
>
> You'd have to audit the C version of LLVM produced by the LLVM backend to spot
> trusting trust exploits in there, because LLVM isn't written in C and needs
> itself to compile itself. (Which is doable, but moderately horrific. Or you
> could audit the resulting binary, which at least nothing _else_ has been built
> with yet so still no circular dependencies in the base OS...)
>
> >> I started cleaning up "man" once and other people were changing it while I was
> >> changing it, so I deleted my changes and merged theirs, and haven't looked at it
> >> again since. Possibly it's quiet enough now, but I haven't cycled back to it.
> >
> > ugh, yeah, i need to dig out my man changes too. unfortunately they're
> > currently in a bit of a local optimum where i need to rewrite quite a
> > bit _again_ now i've uncovered a few more worms.
>
> No rush, but I'm not touching it until it stops moving. :)
>
> >> stdbuf is sort of a wrapper that intercepts a command's stdin/stdout and does
> >> reads/writes of different sizes at it. I keep meaning to give it a closer look
> >> to see if it's worth bothering.
> >
> > does this even work for libcs other than glibc? i thought it just sets
> > an environment variable and execs, with glibc doing the actual work?
>
> Yes and no. At least _some_ of what it does seems to be it reads data and sends
> it along at a different granularity? It's sort of like a line oriented "dd" that
> potentially handles input _and_ output for a child process.
>
> But yeah, there's magic:
>
>   $ env -i /usr/bin/stdbuf -i 0 -o 0 -e 0 $(which toybox) env
>   _STDBUF_I=0
>   _STDBUF_O=0
>   _STDBUF_E=0
>   LD_PRELOAD=/usr/lib/x86_64-linux-gnu/coreutils/libstdbuf.so
>
> I'm not doing an LD_PRELOAD, and if you can't without one I don't think this
> belongs in toybox. :P
>
> >> runcon is also already implemented. It's more selinux nonsense and I haven't got
> >> the selinux libraries installed on my system so the compile time probes make it
> >> drop out, that's why it's in the list. (It's not in _my_ defconfig, but that's a
> >> false negative. :)
> >>
> >> getevent is an android thing: on the one hand elliott hasn't removed it from the
> >> android part of the roadmap (which he maintains at this point), on the other
> >> there isn't one in toybox. *shrug*?
> >
> > i've considered removing it from the list, given that i don't plan on
> > writing a toybox version, and wouldn't necessarily even switch to a
> > toybox version if there was one --- the Android version effectively
> > self-updates based on the kernel headers it's built with, and that's a
> > really nice feature i wouldn't want to lose (and i don't think you
> > want "point toybox to a set of kernel headers" in toybox). we used to
> > update manually, and it was always out of date. (you'd be surprised
> > how many keys/switches/buttons still get added.)
>
> It's not in the debian base install, and the description at:
>
>   https://source.android.com/devices/input/getevent
>
> really sounds like a debugging tool. Removing it might make sense, doesn't seem
> like something toybox really needs.

correct. it's only used by the bringup folks.

> What it REALLY sounds like is there should be a module you can load to get a
> /proc /sys or /dev thing you listen to and get events from (or netlink, or a
> hotplug handler), and userspace doesn't need huge amounts of plumbing to parse a
> text result. (The kernel has lots of config options for human readable output, I
> think the first was KALLSYMS.) "Need to parse version-specific kernel headers to
> build userspace tool" is halfway to "just have the kernel do it already".
>
> *shrug* Not my area...
>
> >> (Of course "make" isn't in the 1.0 roadmap, that's qcc. Getting android
> >> self-hosting through a minimal binary-auditable native build environment is the
> >> NEXT big can of worms. It would be nice if AOSP pulled the NDK as a build
> >> prerequisite, but they're not there yet...)
> >
> > there are inherent circular dependency issues with AOSP and the NDK
> > and LLVM. at the moment the basic flow is LLVM -> AOSP -> NDK (with
> > some of these feeding back too).
>
> Someday, I hope to be in a position to help shovel out the AOSP build
> dependencies, but my plate's overflowing at present.
>
> Rob