[Toybox] Thoughts on seperating shell dependencies and MAYFORK commands?

[Rob Landley]

On 2/21/24 08:29, Mouse wrote:
> I'm not a routine toybox user.  But, to the extent that toybox tries to
> be general-purpose, perhaps some comments from general-purpose shell
> use are relevant here.

"Not a question, more of a comment."

>> There used to be a blog that handed out the "unnecessary use of cat"
>> award, because redirects are cheaper.
> 
> When they work.  I never investigated enough to be in a position to say
> anything definite, but I've seen occasional (emphasis on "occasional")
> cases where cat THING | ... works, but < THING ... doesn't.  Maybe all
> such cases represent bugs?  Not sure.

You didn't give enough information for me to guess why you couldn't get it to work.

>> But "sed 'pattern' long list of input files" is gonna be faster than
>> "for i in long list of input files; do sed $i; done" even if the
>> shell overhead was literally free, because of the repeated setup code
>> in both OS and the command.
> 
> ...again, when both work.

Yes, I know.

> I've done the latter (or more often
> something with xargs) in some cases where the list is so long the
> former would overflow the max arglist size.  I've also (though less
> often) had cases where I don't want to wait for the whole list to be
> computed before getting working on the first few items.

It is often possible to rewrite a "fast on single processor when all data is
immediately available" thing for more parallelism or progressive processing of
gradually delivered input, yes. Which gnu/less keeps breaking again every few years:

  $ while sleep .5; do echo -n .; done | less

I was specifically thinking of the recent 8a2d19bf12c2 commit to scripts/make.sh
header generation where a single call to sed was faster than even parallel calls
to sed launched from a loop, although what I had to give up along the way was
"knowledge of the filename a given block of data lived in" (maybe awk would have
known but I haven't written one of those yet) and instead switch to using the
#define FOR_blah macros to provide structure names generated from GLOBALS() macros.

Rob