[Toybox] [PATCH] file, tar: basic zstd awareness.

[enh]

On Sun, Dec 8, 2024 at 12:51 AM Rob Landley <rob at landley.net> wrote:

> On 12/7/24 18:39, enh wrote:
> > On Sat, Dec 7, 2024, 18:25 Rob Landley <rob at landley.net> wrote:
> >
> >> On 12/6/24 13:57, enh wrote:
> >>> We're seeing ever more zstd-compressed files in the wild, so even
> though
> >>> toybox can't compress/decompress zstd without an external helper, it
> >>> still seems useful to integrate with any that happens to be on the
> >>> system.
> >>
> >> No short option for zstd, even though every other explicit archive
> >> format has one?
> >>
> >
> > technically there are a couple of other compression options that are
> > longopt only,
>
> In gnu/gnu.
>
> > such as --lzma (but i haven't added those here because i've
> > yet to see them used).
> >
> > this probably made sense when it was added in 2019, and it wasn't clear
> how
> > popular, zstd was going to become. (especially in comparison to the other
> > options we don't have.)
> >
> > though tbh, zstd seems more popular in non-tar contexts ... i had to ask
> > the internet what the long and short extensions were!
>
> Imma hijack -Z. I'm aware in debian that's "compress" but we've never
> supported that format, which was patented in the 1980s causing it to be
> completely replaced by gzip except for some old legacy archives you can
> "compress -d file.Z | tar x" if you like.
>

yeah, sounds reasonable.

coincidentally i saw https://www.phoronix.com/news/Linux-EFI-Zboot-Gzip-Zstd
"Linux EFI Zboot Abandoning "Compression Library Museum", Focusing On Gzip
& Zstd" which made me laugh, given that that had been my reaction to the
other formats that gnu tar supports (and has single-letter options for!)
that toybox tar doesn't (and almost certainly shoudn't) like lzip and lzop.
presumably characters from a children's show in a language i don't speak?


> (I just like there to BE a short option, and another obvious contender
> isn't presenting itself. Plus I haven't got an obvious way to test this
> anyway.)
>

yeah, i just tested manually. it did occur to me that the test shell script
could check to see whether there's a zstd(1) binary on the path, and skip
any zstd tests if not?

(and there's really no excuse for me not adding a file(1) test beyond "we
don't have tests for _most_ of the recognized formats", though "this is
just a constant prefix match" is a slightly better excuse.)


> Hmmm...
>
> https://www.rfc-editor.org/rfc/rfc8878.txt
>
> And according to https://github.com/facebook/zstd the "reference
> implementation" is dual licensed BSD and GPL. How bad is...
>
> https://github.com/facebook/zstd/blob/dev/lib/legacy/zstd_legacy.h
>
> Nope, that's terrifying. But:
>
> https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md
>
> Looks possibly doable.
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20241210/a4b26ec9/attachment.htm>