[Toybox] "bad xform" not very helpful

[David Seikel]

On 2022-08-31 14:55:24, enh via Toybox wrote:
>    On Tue, Aug 16, 2022 at 12:22 PM enh <enh at google.com> wrote:
> 
>      On Tue, Aug 16, 2022 at 10:28 AM enh <enh at google.com> wrote:
> 
>        On Tue, Aug 16, 2022 at 1:43 AM Rob Landley <rob at landley.net> wrote:
> 
>          On 8/15/22 18:50, enh via Toybox wrote:
>          > and here's their minimized repro case:
>          >
>          > echo > /tmp/foo.txt; echo /tmp/foo.txt > /tmp/find.txt
>          >
>          > cat /tmp/find.txt | prebuilts/build-tools/path/linux-x86/tar czf
>          /tmp/out.tar.gz \
>          >   --absolute-names \
>          >   --transform 's,^/,,' -T -
>          >
>          > This fails with
>          >
>          > tar: bad xform
> 
>          Hmmm...
> 
>          $ echo > /tmp/foo.txt; echo /tmp/foo.txt > /tmp/find.txt
>          $ cat /tmp/find.txt | PATH=$PWD/sub9:$PATH ./tar czf out.tar.gz \
>            --absolute-names --transform 's,^/,,' -T -
>          $ tar tvf /tmp/out.tar.gz
>          -rw-r--r-- landley/landley   1 2022-08-16 01:53 tmp/foo.txt
> 
>          Working for me? (The sub9 bit was because I stuck toybox sed in the
>          $PATH to
>          make sure that wasn't it...)
> 
>        repos for me, both with their prebuilt but also with a fresh clone (on
>        either macos or linux):
> 
>        /tmp/toybox$ cat /tmp/find.txt | ./toybox tar czf /tmp/out.tar.gz  
>        --absolute-names   --transform 's,^/,,' -T -
> 
>        tar: bad xform
> 
>        /tmp/toybox$ 
> 
>         
>        a bit of printf debugging shows we're reading nothing back:
> 
>        /tmp/toybox$ cat /tmp/find.txt | strace -f ./toybox tar czf
>        /tmp/out.tar.gz   --absolute-names   --transform 's,^/,,' -T - 2>
>        /tmp/out
> 
>        argv[0]="sed"
> 
>        argv[1]="-e"
> 
>        argv[2]="s,^/,,"
> 
>        pid=1779946
> 
>        stdin="/tmp/foo.txt"
> 
>          len=0 Success
> 
>        total=0 result="(null)"
> 
>        but strace implies we're not actually exec()ing sed at all?
> 
>      and if i `CONFIG_TOYBOX_NORECURSE=y`, it calls sed and works...
> 
>    ...though this might be about to come back and bite me. i'm hearing as-yet
>    unconfirmed reports that toybox `tar czf` is a lot slower than gnu tar,
>    and -- given that they're using --transform` while they're assuming it's
>    tar or gzip, i'm wondering whether it's actually the fact that we're
>    forking out to sed for every file?
>    i've asked for repro steps or a `perf record` i can look at...

That reminds me...

I've long thought it would be great if toybox toys could call other
toybox toys directly.  So tar wouldn't fork out to sed, we have our own
sed, just call it directly.  Then you avoid the fork and other overhead,
which will probably speed things up.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.