[Toybox] [PATCH] tar: add -I (--use-compress-program) support.

[enh]

On Tue, Aug 25, 2020 at 1:20 AM Rob Landley <rob at landley.net> wrote:
>
> On 8/24/20 12:15 PM, enh via Toybox wrote:
> > unfortunately, -I was just the easy request... they also want --transform.
> >
> > this is the specific command:
> >
> > tar -I $(KGZIP) -c $(RCS_TAR_IGNORE) -f $(2).tar.gz \
> >     --transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3); \
> >
> > https://android.googlesource.com/kernel/common/+/refs/heads/android-mainline/scripts/Makefile.package#49
>
> Eh, this is just some variant of:
>
> int fds[2] = {-1, -1}, len;
> pid_t pid = xpopen_both((char *[]){"sed", "-e", arg, 0}, fds);
>
> if (pid<0) sadness();
> for (;;) {
>   char *line = 0;
>
>   len = 0
>   dprintf(*fds, "%s\n", filename);
>   len = readline(&line, *len);
> }
>
> > it's not clear to me just how complete that "sed expression" is. oh, looks like
> > it's just 's': https://www.gnu.org/software/tar/manual/tar.html#SEC115 (but with
> > some weird extra flags rRsShH)
>
> Except for that bit, where it's using flags not explained in the manual. I don't
> have "info" installed, is this available on the web somewhere?
>
> https://www.gnu.org/software/sed/manual/sed.html#The-_0022s_0022-Command
>
> Does not mention s///S because of course not.
>
> sed s///H is multi-line mode? Is there any way I can continue to _not_ care
> about that for a little while longer? Hmmm... \S is a regex extension to match
> non-whitespace chars but there's no \ here...
>
>   $ echo hello | sed -e 's/x/y/S'
>   sed: -e expression #1, char 7: unknown option to `s'
>
> And the gnu/dammit sed in debian doesn't understand trailing S either. Huh. Is
> it a tar --transform thing? Nothing in the man page about that...
>
> Ok, it IS a tar thing (not a sed thing) which is NOT described in the tar man
> page but IS described in the tar online manual but NOT in the part about
> --transform but instead in the "modifying file and member names" part and then
> hit page down 4 times.

yeah, for those following along at home:
https://www.gnu.org/software/tar/manual/html_section/tar_51.html

> Wheee.
>
> Do I police that the sed pattern is s/x/x/ or just let them pass in y/x/z and
> friends if they really want to? (Hmmm, probably police it given they can w to
> create new files otherwise. So start with s%c and then require the %c twice more
> and no ; after the last %c. No ispunct at all, really...
>
> ok, gix#rRsShH are the supported flags. Currently gi# are normal sed commands
> (along with iI). We don't yet support x and that seems like basic sed should
> have that but it's not in the sed manual. (I can feed -E to sed when we see x
> but controlling that for a given s/// search pattern seems kinda useful. Oh well...
>
> rRsShH are just "apply to regular, symlink, or hardlink" target filters. Easy
> enough to put in the

i don't think this works given the fact that you can have multiple s
commands (though that's almost unused in debian:
https://codesearch.debian.net/search?q=tar.*--transform.*%3B&literal=0
), each with different flags (both sed style and "what am i applicable
to?" flags), plus the weird flags= thing (which is also almost unused
in debian: https://codesearch.debian.net/search?q=tar.*--transform.*flags%3D&literal=0
).

i personally only need 's:^:prefix/:S' but the vast majority of debian
uses also seem to be just one s/pattern/replacement/flags (
https://codesearch.debian.net/search?q=tar.*--transform.*&literal=0 ),
i assume that's a good enough place for me to start? (it wasn't clear
to me whether when you said "no ; after the last %c" you meant
"because i don't think we should support multiple s commands unless
proven necessary" or "because i'm not aware that GNU tar supports
multiple s commands".)

btw, strace shows that GNU tar doesn't spawn a sed process for this.
but if we're only doing the simplest case, i think we can get by with
the pipe.

> Anyway, yeah should be doable.
>
> Rob
> _______________________________________________
> Toybox mailing list
> Toybox at lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net