[Toybox] "bad xform" not very helpful

[enh]

On Wed, Aug 31, 2022 at 11:37 PM Rob Landley <rob at landley.net> wrote:

> On 8/31/22 16:55, enh wrote:
> >     and if i `CONFIG_TOYBOX_NORECURSE=y`, it calls sed and works...
> >
> > ...though this might be about to come back and bite me. i'm hearing
> as-yet
> > unconfirmed reports that toybox `tar czf` is a lot slower than gnu tar,
> and --
> > given that they're using --transform` while they're assuming it's tar or
> gzip,
> > i'm wondering whether it's actually the fact that we're forking out to
> sed for
> > every file?
> >
> > i've asked for repro steps or a `perf record` i can look at...
>
> When I said I had more work to do there, part of what I wanted to do was
> add a
> sed --tar-xform mode with some sort of protocol that would allow a single
> persistent sed to get a name and send back a response without exiting.
> (That
> would also allow me to implement the trailing type indicators.)
>
> Right now exiting lets you know that it's done processing, even if your sed
> invocation included a "d" or something and thus produced no output (so you
> wait
> forever if it DOESN'T exit). And even if I use -z mode you can include \0
> manually in a substitution and thus force it out of sync. So the
> persistent mode
> would need some sort of "expect X bytes of output" indicator, probably
> just a
> decimal number of bytes ending with a newline, followed by the transformed
> payload. And of course the type indicator needs the incoming name to be
> annotated with type information as part of the call-and-response
> protocol...
>
> It's not perfect, you can always sed ':a;b a' or something if you want to
> cause
> Horrible Goose levels of trouble. (Tempted to add something like the
> symlink
> resolution limit so 1 billion jumps in a single sed pattern evaluation
> exit with
> an error...)
>
> The downside of doing that is it would require TOYBOX sed.


meh, _i_ don't care about that :-)

(and, more seriously, does _anyone_ care about using a mixture of toybox
and non-toybox? i'd imagine the only people who'd even notice would be
people like me [or these kernel guys] trying to switch over, and the only
reason i/they are switching over one tool at a time is because we've been
running slightly ahead of you, using stuff that's not finished yet. i guess
it would affect the toybox _tests_ but i kind of thing that's a bug already
--- given that i support systems where everything's from toybox it's a
misfeature that the toybox tests _don't_ test toybox-with-toybox [and i get
no value from them testing toybox-with-gnu].)


> Right now it should
> work with the host sed. I vaguely plan to try to launch "sed --tar-xform -e
> $PATTERN" and then fall back to the old one at a time logic not
> undestanding
> suffixes if sed exits with an error, because toybox commands try NOT to
> require
> other toybox commands. But I can't figure out a better way to do this than
> copying half the sed plumbing to lib/sed.c and it really wasn't designed
> to be
> broken up...
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20220901/35bb6ae2/attachment.htm>