[Toybox] Thoughts on seperating shell dependencies and MAYFORK commands?

[Rob Landley]

On 2/20/24 19:41, enh wrote:
>> The SHELL won't do it, because the shell cares fairly deeply about what is and
>> isn't a child process. In fact pipelines have implicit ( ) around each entry
>> because each one is a subshell to avoid pipelines blocking on full pipe buffers.
>> (And yes that includes the last one in the list, for consistency. And thus "echo
>> | x=47; echo $x" isn't going to remember the 47, same as bash.)
>>
>> But on an mmu system, fork() is like 5% of the expense and exec() is 95%.
> 
> (in case you ever get quoted out of context, i'll add "for a process
> like toybox that doesn't have threads, large numbers of VMAs, or large
> numbers of open fds" here. because those things do make fork()
> expensive in large systems. but, no, not for toybox.)

Oh sure. https://landley.net/notes-2018.html#26-06-2018 for example. :)

>> There are a couple exceptions, like true/false which are SO cheap that the extra
>> overhead from fork() is noticeable in "while true; do blah; done" loops. And
>> "echo" is historically a bash builtin so a standalone shell on a system with no
>> $PATH might want that. But as you've pointed out, even cat didn't get that
>> annotation.
> 
> (fwiw, mksh _does_ have a builtin cat. that's something i disabled on
> Android because it was confusing to users. for "fully toybox" systems
> though, with just one binary and loads of symlinks, you wouldn't be
> able to tell the difference as easily. though iirc it was weird
> signal-related behavior that was the first time someone noticed cat
> was a builtin?)

If there's an actual _need_ for cat to be a builtin it can be, but I hadn't
noticed a demand?

There used to be a blog that handed out the "unnecessary use of cat" award,
because redirects are cheaper. Once upon a time I had a todo item to try to work
splice(2) into more stuff, which gave over to sendfile(2) instead which
xsendfile() and friends hopefully cover today, but the fundamental problem is
the process context can't go away even after we shorted two filehandles together
so data goes straight from one to the other. Even when the process context no
longer has anything to do, it can't exit. (Zombie Prime.)

I asked about this on lkml long ago, noting that technically you can pass a
filehandle across a unix domain socket and see what another process's file
descriptors are connected to in /proc so you could KIND of implement this in
userspace with ptrace if you really tried, so it shouldn't be THAT hard for the
kernel to do, and the resulting screams of conceptual design-level anguish
dissuaded me from pursuing it further...

(Alas, the old "there is no way to do this" -> "here's an utterly terrible way
to do it" -> "don't ask questions post errors" -> "better fix ASAP to prevent
THAT" pipeline stopped being acceptable on linux-kernel roughly around the 2008
mortgage crisis. Also, in this case, something about reference counting?)

I _could_ also have the shell special case recognize "cat singlefile |" as an
input redirect, maybe that's what mksh is doing? But I really dowanna unwind
"cat <(potato) |" and friends. Hmmm, speaking of which...

$ bash -c 'cat <<< <(echo hello)'
/dev/fd/63
$ toybox sh -c 'cat <<< <(echo hello)'
<(echo hello)
$ toybox sh -c 'echo <(echo hello)'
/proc/self/fd/3

Ok, I don't even know what "fixing" that would look like. Throw it on the todo heap.

>> Seriously, benchmark it. The expense isn't clone(2), it's execve(2). (Now maybe
>> selinux nonsense makes that go weird, couldn't say...)
> 
> (as a libc maintainer, "the expense is all in your ELF constructors"
> :-)

As we discovered, MacOS homebrew launch time is horrific. But even there, you
only really notice this in loops. We're talking fractions of a second here.

But "sed 'pattern' long list of input files" is gonna be faster than "for i in
long list of input files; do sed $i; done" even if the shell overhead was
literally free, because of the repeated setup code in both OS and the command.

> last i looked, most of the "wasted" time in a trivial toybox
> invocation on Android is some front-loaded work in the networking code
> that makes sense for the zygote but would be unfortunate for someone
> running lots of random unix commands!)

I thought the zygote ran some setup code before it spins forking itself?
(https://developer.android.com/topic/performance/memory-overview says "common
framework code and resources" which could mean anything). I wonder why that
network setup's in libc instead of the zygote's main()...

Rob