[Toybox] "bad xform" not very helpful

[Rob Landley]

On 8/31/22 16:55, enh wrote:
>     and if i `CONFIG_TOYBOX_NORECURSE=y`, it calls sed and works...
> 
> ...though this might be about to come back and bite me. i'm hearing as-yet
> unconfirmed reports that toybox `tar czf` is a lot slower than gnu tar, and --
> given that they're using --transform` while they're assuming it's tar or gzip,
> i'm wondering whether it's actually the fact that we're forking out to sed for
> every file?
> 
> i've asked for repro steps or a `perf record` i can look at...

When I said I had more work to do there, part of what I wanted to do was add a
sed --tar-xform mode with some sort of protocol that would allow a single
persistent sed to get a name and send back a response without exiting. (That
would also allow me to implement the trailing type indicators.)

Right now exiting lets you know that it's done processing, even if your sed
invocation included a "d" or something and thus produced no output (so you wait
forever if it DOESN'T exit). And even if I use -z mode you can include \0
manually in a substitution and thus force it out of sync. So the persistent mode
would need some sort of "expect X bytes of output" indicator, probably just a
decimal number of bytes ending with a newline, followed by the transformed
payload. And of course the type indicator needs the incoming name to be
annotated with type information as part of the call-and-response protocol...

It's not perfect, you can always sed ':a;b a' or something if you want to cause
Horrible Goose levels of trouble. (Tempted to add something like the symlink
resolution limit so 1 billion jumps in a single sed pattern evaluation exit with
an error...)

The downside of doing that is it would require TOYBOX sed. Right now it should
work with the host sed. I vaguely plan to try to launch "sed --tar-xform -e
$PATTERN" and then fall back to the old one at a time logic not undestanding
suffixes if sed exits with an error, because toybox commands try NOT to require
other toybox commands. But I can't figure out a better way to do this than
copying half the sed plumbing to lib/sed.c and it really wasn't designed to be
broken up...

Rob