[Toybox] [PATCH] Clean up xz a good amount

[Rob Landley]

On 3/29/24 17:28, enh wrote:
> On Wed, Feb 28, 2024 at 9:13 AM enh <enh at google.com> wrote:
>> > > @@ -639,6 +640,20 @@ enum xz_ret xz_dec_bcj_run(struct xz_dec_bcj *s, struct xz_dec_lzma2 *lzma2,
>> > >   */
>> > >  enum xz_ret xz_dec_bcj_reset(struct xz_dec_bcj *s, char id)
>> > >  {
>> > > +  switch (id) {
>> > > +  case BCJ_X86:
>> > > +  case BCJ_POWERPC:
>> > > +  case BCJ_IA64:
>> > > +  case BCJ_ARM:
>> > > +  case BCJ_ARMTHUMB:
>> > > +  case BCJ_SPARC:
>> > > +    break;
>> > > +
>> > > +  default:
>> > > +    /* Unsupported Filter ID */
>> > > +    return XZ_OPTIONS_ERROR;
>> > > +  }
>> > > +
>> > >    s->type = id;
>> > >    s->ret = XZ_OK;
>> > >    s->pos = 0;
>>
>> ah, crap, that's another thing to put on the riscv64 to-do list...
>> (thanks for bringing that to light!)
> 
> so, TIL that upstream already added a risc-v bcj implementation...

I'm happy to call the public domain repo our "upstream" for this, but there's
still some collation damage (they have many files and we want either one or
two), and a lot of cleanup that could be done in our code that moves it farther
from their code.

As for whether we want one file or two: one model is the engine in the command
ala toys/*/bzcat.c and the other is lib/deflate.c called by toys/*/gzip.c (but
also available for other things to pull in without having to fork a child
process and pipe data through it). But the real difference there is deflate has
half an inflate already that I REALLY SHOULD FINISH (dictionary selection and
resets, everything else is just a question of doing the work) and xz compression
seems a bit out of scope. (Being able to read everything: yay. Being able to
compress data, gzip is the 80/20.

Modulo busybox refuses to build without bzip2 compression (I hit it until it
confessed in mkroot/packages/busybox.c but that broke all the help text), and I
did WRITE a cleaned up bzip2 engine many moons ago (reposted it here not to long
ago I think), so I _could_ have a lib/bzip2.c with a compression side if I
wanted to? Modulo the bzip2 compression side string sort logic never made sense
to me (what is the logic of falling back from one sort mechanism to the next,
why those in that order with those thresholds) so to test my engine I had to
block copy the original sort logic, which has licensing issues...

> ...but i only learned that because i was looking at
> https://www.openwall.com/lists/oss-security/2024/03/29/4 which was
> fascinating in many ways.
> 
> (rob will of course be delighted to hear of systemd's involvement in
> the exploit chain :-) )

I saw a youtube video on it, and it's been all over mastodon today. So much
unnecessary complexity. Adding layers to "solve" problems is painting over dry
rot. There are reasons I also want to simplify the build system itself, and care
so much about comparing the behavior across multiple platforms...

Rob