[Toybox] New email, [PATCH] fdisk cleanup, etc.

[Rob Landley]

On 08/12/2013 12:30:05 AM, Isaac wrote:
> On Sun, Aug 11, 2013 at 10:07:49PM -0500, Rob Landley wrote:
> > On 08/11/2013 07:42:37 PM, Isaac wrote:
> > >I also have some code here that *should* handle writing a member
> > >of a cpio
> > >newc archive, loopfiles_stdin(), and a general idea of how to  
> proceed.
> > >Don't expect much soon.
> > >(newc is essentially an octal dump of struct stat; oldc differs
> > >only in
> > >having a few fields missing or differently sized. cpio is
> > >well-suited to
> > >loopfiles, except it reads from stdin.
> >
> > I've only been looking at newc, because that's what initramfs uses.
> > If RPM (the other major modern user of cpio I'm aware of) also uses
> > newc, then that's probably all we need to implement. (Dunno about
> > full rpm support, but we should be able to do rpm2cpio.)
> 
> http://people.freebsd.org/~kientzle/libarchive/man/cpio.5.txt
> https://github.com/ereshetova/rpm/blob/master/lib/cpio.h
> 
> Yes, that's what RPM uses.
> mksh used to be distributed in cpio odc format, which really doesn't
> take all that different code. They use some tar format now.

It's like patch only doing -u: there are files out there that _aren't_  
in that format, but there are other tools if you really need that.

> By the way: for extracting CPIO archives, if nlink > 1, check  
> devmajor,
> devminor, and ino. If they have not been recorded, record and extract;
> if they have, create a hard link.
> I'm guessing you'd need something vaguely like this:
> struct list_members{

Sigh. I've been meaning to do some variant of a really simple hash  
table or balanced tree. (More likely a tree.) Tar needs hardlink  
checking too, and the way I want to write gene2fs that's an archiver as  
well. (It's a kind of annoying archiver because you need to enumerate  
all the files and work out where you're going to store them before you  
can write the initial metadata; this is why zip sticks its metadata on  
the end, but I never got around to writing a "mount -t zip" driver  
before squashfs went in.)

Anyway; I'm uncomfortable with an n^2 data storage/lookup mechanism on  
an unbounded data set. Try to cpio a gigabyte directory and it could  
get really slow. (Hmmm... but if it only cares about link>1...)

> struct list_members *next;
> struct list_members *prev;
> int orig_ino; //24 bits saved
> int nlink; // decrement by 1 on every write matching file?
> dev_t orig_dev; //devmajor + devminor on newc/"SVR4"
> long new_ino;
> dev_t new_dev;
> }
> 
> 
> > There was a really fun trick somebody suggested last decade in
> > busybox, which was to grab the metadata header part of each rpm file
> > you install and dump it in a directory, then just do the rpm -qpi
> > code to read what's installed. (So no database, just a directory of
> > chunks of the files.)
> 
> I remember seeing that discussion.
> By the way, this reminds me of Mastodon Linux ("as free of GNUware as
> is practical with Linux"), xpkg, and makepkg.

I'm trying to make that quote "100%". Working on it...

> http://www.mastodon.biz/
> http://www.pell.portland.or.us/~orc/Code/
> http://www.pell.portland.or.us/~orc/Code/xrpm/
> ^This is where xpkg/makepkg come from. They can create or extract RPM
> packages, or at least some version thereof.

I've been keeping very, very vague tabs on  
https://lwn.net/Articles/441085/

At timesys, they did cross compiling with a Horrible Perl Wrapper  
around rpm, which did some LD_PRELOAD magic, command line rewriting,  
and more environment variables than you can possibly imagine.  
(Hopefully tsrpm has died a well-deserved death by now; my relaunch of  
aboriginal was aimed at replacing exactly that sort of thing.)

That said, rpm does a lot of stuff that I'm not entirely clear on. I'm  
looking at RPM first instead of dpkg first because Red Hat paid off the  
linux foundation's predecessor to put RPM in the LSB (to the eternal  
annoyance of Ubuntu, but what do you _expect_ once you've written  
alien? It's like win-OS/2 guaranteeing nobody would write any more OS/2  
native software). But unfortunately, LSB doesn't specify how to  
_implement_ a conformant RPM, just says you should have one.

(The Linux Standard Base hopes that someday it can grow up to be a real  
standards body, but absent a blonde woman in a blue gown floating down  
with a wand, it ain't happening. The closest they've managed is a dark  
haired guy in a business suit on a lot of airplanes with a cell phone,  
and it's just not the same.)

> (Coincidentally, he ended up writing a mail forwarder because that was
> simpler than getting configure to run on one platform he was using.)

I'm leaning that way with mail clients, myself...

> > The main reason we never bothered was all the install script
> > nonsense that can do arbitrary turing complete stuff at install and
> > uninstall time. I just basically never got around to it...
> >
> > >I'm working on cpio instead of pax because (a) I want to write an
> > >initrd,
> > >and (b) pax has a much more complex specification).
> >
> > I was only planning on doing tar, not pax. I'm aware Solaris got it
> > into posix, but I don't care. Almost nobody uses it on Linux,
> > everybody uses tar instead.
> 
> /me raises hand sheepishly
> 
> I use it on occasion; mainly because it's MirBSD pax vs. GNU cpio
> and GNU tar.  pax supports all the formats (except the oddball ancient
> Solaris and V7 tarballs?), is smaller than either, and doesn't have
> builtin support for reading and writing files and tapes on remote  
> servers
> (seriously: what else do you expect of *GNU* cpio and tar?)
> And it's got a meaningful manpage.

If you want to do "pax" as an alternate command line for tar, go for  
it. (Alas, OLDTOY still doesn't do this. I need to sit down and think  
about how to handle that.) But that's optional; tar is required.

Pax is posix, but tar is Linux. All the actual files out there are  
*.tar, that's what busybox cloned, it's what Ubuntu and Knoppix install  
by default (neither has pax, both have tar). Thus having the "tar"  
command line able to extract those files is important to me, and having  
pax isn't.

Oddly, Red Hat 9 from 10 years ago seems to have installed pax by  
default. (I don't have a current fedora, sles, or gentoo set up, need  
to reinstall them when I finish moving to the new laptop, which is held  
up by not wanting to reinstall balsa, not yet finding a better email  
client, and not having worked up the apathy to reinstall Thunderbird  
wot done me wrong). And the fact that Ubuntu hasn't got pax is less  
convincing when you realize ubuntu hasn't got _patch_ installed by  
default either. But I have yet to find a Linux system that has pax but  
_not_ tar.

> On the other hand, I tremble at the thought of GNU deciding to  
> implement
> pax.

I wouldn't pay the slightest bit of attention of they did: there's a  
posix page on it, and a gazillion existing tarfiles with fun things  
like out-of-band git metadata in them.

Parse the files that exist, create files other tools can read, wait for  
complaints. :)

Rob