<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 16, 2022 at 12:22 PM enh <<a href="mailto:enh@google.com">enh@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 16, 2022 at 10:28 AM enh <<a href="mailto:enh@google.com" target="_blank">enh@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 16, 2022 at 1:43 AM Rob Landley <<a href="mailto:rob@landley.net" target="_blank">rob@landley.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 8/15/22 18:50, enh via Toybox wrote:<br>
> and here's their minimized repro case:<br>
> <br>
> echo > /tmp/foo.txt; echo /tmp/foo.txt > /tmp/find.txt<br>
> <br>
> cat /tmp/find.txt | prebuilts/build-tools/path/linux-x86/tar czf /tmp/out.tar.gz \<br>
>   --absolute-names \<br>
>   --transform 's,^/,,' -T -<br>
> <br>
> This fails with<br>
> <br>
> tar: bad xform<br>
<br>
Hmmm...<br>
<br>
$ echo > /tmp/foo.txt; echo /tmp/foo.txt > /tmp/find.txt<br>
$ cat /tmp/find.txt | PATH=$PWD/sub9:$PATH ./tar czf out.tar.gz \<br>
  --absolute-names --transform 's,^/,,' -T -<br>
$ tar tvf /tmp/out.tar.gz<br>
-rw-r--r-- landley/landley   1 2022-08-16 01:53 tmp/foo.txt<br>
<br>
Working for me? (The sub9 bit was because I stuck toybox sed in the $PATH to<br>
make sure that wasn't it...)<br></blockquote><div><br></div><div>repos for me, both with their prebuilt but also with a fresh clone (on either macos or linux):</div><div><br></div><div>





<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(170,171,37)"><b>/tmp/toybox$ </b></span><span style="font-variant-ligatures:no-common-ligatures">cat /tmp/find.txt | ./toybox tar czf /tmp/out.tar.gz <span>  </span>--absolute-names <span>  </span>--transform 's,^/,,' -T -</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">tar: bad xform</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(170,171,37)"><span style="font-variant-ligatures:no-common-ligatures"><b>/tmp/toybox$<span> </span></b></span></p></div><div> </div><div>a bit of printf debugging shows we're reading nothing back:</div><div><br></div><div>





<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(170,171,37)"><b>/tmp/toybox$ </b></span><span style="font-variant-ligatures:no-common-ligatures">cat /tmp/find.txt | strace -f ./toybox tar czf /tmp/out.tar.gz <span>  </span>--absolute-names <span>  </span>--transform 's,^/,,' -T - 2> /tmp/out</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">argv[0]="sed"</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">argv[1]="-e"</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">argv[2]="s,^/,,"</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">pid=1779946</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">stdin="/tmp/foo.txt"</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span>  </span>len=0 Success</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">total=0 result="(null)"</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><br></p></div><div>but strace implies we're not actually exec()ing sed at all?</div></div></div></blockquote><div><br></div><div>and if i `CONFIG_TOYBOX_NORECURSE=y`, it calls sed and works...</div></div></div></blockquote><div><br></div><div>...though this might be about to come back and bite me. i'm hearing as-yet unconfirmed reports that toybox `tar czf` is a lot slower than gnu tar, and -- given that they're using --transform` while they're assuming it's tar or gzip, i'm wondering whether it's actually the fact that we're forking out to sed for every file?</div><div><br></div><div>i've asked for repro steps or a `perf record` i can look at...</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div> </div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>let me know if you've already fixed this on your branch and that's why you can't repro, otherwise i'll keep looking after my meeting...</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> However, if the file names are fed via the -T /tmp/find.txt, it works:<br>
<br>
Hmmm... the child process shouldn't have access to the parent's stdin, we<br>
replaced it with a pipe? There was a potential bug in that area, but commit<br>
dc8b46d5ddab should have fixed it last month and I don't _think_ it would have<br>
applied here anyway...<br>
<br>
> echo > /tmp/foo.txt; echo /tmp/foo.txt > /tmp/find.txt<br>
> <br>
> prebuilts/build-tools/path/linux-x86/tar czf /tmp/out.tar.gz \<br>
>   --absolute-names \<br>
>   --transform 's,^/,,' -T /tmp/find.txt<br>
> <br>
> (the "prebuilts/build-tools/path/linux-x86/" stuff is just a directory full of<br>
> symlinks to toybox.)<br>
<br>
Multiplexer instead of standalone build shouldn't make a difference if you've<br>
disabled command recursion. (Modulo you're calling tar out a specific path but<br>
it then grabs sed out of the $PATH, but I haven't yet implemented the extra<br>
argument processing that would specifically require toybox sed...)<br>
<br>
(The extra error message is a little tricker than my first guess because you can<br>
have multiple --xform things which turn into a list of -e entries to sed...<br>
Possibly instead of error_exit() I should error_msg(), dump the sed command line<br>
on a second line, and then xexit()...)<br>
<br>
Rob<br>
</blockquote></div></div>
</blockquote></div></div>
</blockquote></div></div>