<div dir="ltr"><div>(I'll try to keep it short)</div><div><br></div><div>My original motivation to send this patch is that my coworker found out (when he was working with initramfs) that "cpio -u" behaves differently on toybox compared to the GNU's implementation.</div><div>(my original email, in case it was flagged as spam...: <a href="http://lists.landley.net/pipermail/toybox-landley.net/2021-February/012270.html">http://lists.landley.net/pipermail/toybox-landley.net/2021-February/012270.html</a>)</div><div>In short, I found that toybox cpio doesn't exactly behave like "-u" is always on. I'll write the repro steps here again, since I discovered another oddity when "-d" is specified:</div><div><br></div><div>(testing toybox cpio)</div><div>$ mkdir a</div><div>$ echo "a" | cpio -H newc -o >a.cpio</div><div dir="ltr">$ toybox cpio -iu <a.cpio</div><div dir="ltr">cpio: a: File exists</div><div dir="ltr"><br></div><div>(toybox shouldn't report EEXIST if the directory to be created already exist)</div><div dir="ltr"><br></div>$ echo "a/" | cpio -H newc -o >a.cpio<div>$ rm -rf a</div><div>$ toybox cpio -iud <a.cpio<br>cpio: a/: File exists<br><br>("-d" creates the leading directory of "a/", which is "a", which causes cpio to complain that "a" already exists when it tries to mkdir("a/")...)</div><div><br></div><div>My goal was to make the two cases above to not report any error (the patch I sent only addressed the first case though),</div><div>and a follow-up question is that do we implement different behaviors with and without "-u"?</div><div>I'm not trying to make the behavior of toybox cpio match other commands or implementations.</div><div>Though it's a welcome change IMO if it makes subcommands behave more consistently. For example making toybox cpio behave similar to toybox tar.</div><div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Feb 28, 2021 at 6:41 PM Rob Landley <<a href="mailto:rob@landley.net">rob@landley.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 2/27/21 10:18 AM, Yi-yo Chiang wrote:<br>
> Using different command to create the archive yields different result, I used $<br>
> find dir | cpio -o ... to create the archive for testing:<br>
> <br>
> (testing the behavior of GNU cpio)<br>
> $ mkdir a<br>
> $ echo blah > a/bb<br>
> $ find a | cpio -H newc -o >a.cpio<br>
> $ # Archive a.cpio creates a (dir) and a/bb (file)<br>
<br>
It triggers on an exact match but not a mkpath. So the "create leading<br>
directory" plumbing doesn't do it, only if the directory is in the archive<br>
before the file contained in it.<br>
<br>
Were the gnu devs lazy and never bothered to properly implement -d combined with<br>
-u, or is this an intentional "design" difference?<br></blockquote><div><br></div><div>I guess it's intentional? So doing something like this don't accidentally remove "a" when the intention was to create "a/bb":</div><div>$ mkdir a</div><div>$ echo blah >a/bb</div><div>$ find a | cpio -H newc -o >a.cpio</div><div>$ rm -rf a</div><div>$ touch a</div><div>$ cpio -idu a/bb <a.cpio</div>cpio: `a' exists but is not a directory<br>cpio: a/bb: Cannot open: Not a directory<br>1 block<br><div><br></div><div>(just my speculation)</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> $ # Test dir from archive replaces existing file<br>
> $ rm -rf a<br>
> $ touch a<br>
> $ cpio -i <a.cpio<br>
> cpio: a not created: newer or same age version exists<br>
<br>
That's not something tar or zip do. Why does cpio do it? Does anybody depend on<br>
cpio doing it?<br></blockquote><div><br></div><div>Probably because that was what the spec said? Quote <a href="https://pubs.opengroup.org/onlinepubs/7908799/xcu/cpio.html">https://pubs.opengroup.org/onlinepubs/7908799/xcu/cpio.html</a> : "option: u: Copy unconditionally (normally, an older file will not replace a newer file with the same name)."</div><div>I too would want to know who depends on this behavior. I only ever encounter cpio archives when dealing with initramfs, and as far as I know initramfs always replaces like "cpio -u".</div><div>Perhaps there are some use cases out there that use "cpio -p" to sync files, thus depending on this rsync-esque timestamp checking behavior? </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
My original idea of ignoring -u was to just do the -u behavior all the time,<br>
because that's what tar does. You're instead making it so not supplying -u never<br>
replaces files, and... I'm not sure why?<br></blockquote><div><br></div><div>I assume you missed <a href="http://lists.landley.net/pipermail/toybox-landley.net/2021-February/012270.html" class="cremed">my first email</a>. Actually I agree we make "-u" the default and always replace everything (for the sake of simplicity), and I was just tossing out the patch to spark some discussion!</div><div>It's also pretty straight-forward to modify my patch to make it ignore "-u" and make "-u" the default. Just remove the part surrounded by "<span style="color:rgb(0,0,0);white-space:pre-wrap">if (!FLAG(u)) {...}</span>".</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
According to<br>
<a href="https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt" rel="noreferrer" target="_blank" class="cremed">https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt</a><br>
(which I wrote so long ago the behavior was actually different at the time and I<br>
had to update it, but presumably if it was no longer accurate somebody would<br>
have committed a patch?) says that the kernel initramfs plumbing will always<br>
overwrite. (The original behavior is that it would append the new file's<br>
contents to the old file, which they decided they didn't like. :)<br></blockquote><div><br></div><div>(this whole thread actually all started because I and my coworker are working with initramfs... I did read that doc beforehand, but I never notice the author was you until now!)</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> cpio: a/bb: Cannot open: Not a directory<br>
> 1 block<br>
> $ # Failed to create a (dir)<br>
> $ touch -d yesterday a<br>
> $ cpio -i <a.cpio<br>
> 1 block<br>
> $ # Interesting, if the timestamp of a (file) is older than the archive's a<br>
> (dir), then replacement happens<br>
> $ touch a<br>
> $ cpio -iu <a.cpio<br>
> 1 block<br>
<br>
Would always replacing and not doing this weird timestamp comparison break<br>
anything you know of?<br></blockquote><div><br></div><div>Nothing I can think of, other than the hypothetical use case of "cpio -p" I came up with above.</div><div>But I also won't be surprised if some decades old script may depend on this... :/</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> $ # -u unlinks a (file) then mkdir(a)<br>
> <br>
> $ # Test file from archive replaces existing empty dir<br>
> $ rm a/bb<br>
> $ mkdir a/bb<br>
> $ cpio -i <a.cpio<br>
> cpio: a/bb not created: newer or same age version exists<br>
> 1 block<br>
> $ touch -d yesterday a/bb<br>
> $ cpio -i <a.cpio<br>
> 1 block<br>
> $ rm a/bb<br>
> $ mkdir a/bb<br>
> $ cpio -iu <a.cpio<br>
> 1 block<br>
> $ # Overall similar result when replacing existing file.<br>
<br>
What portion of this is new? What are you trying to prove?<br></blockquote><div><br></div><div>I'm showing that if "a/bb" exist as an empty directory, the archive contains a regular file "a/bb", and the existing directory has an older timestamp, then cpio would rmdir(a/bb) before extracting "a/bb" (file).</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> $ # Test when a/bb is not empty<br>
> $ rm -rf a<br>
> $ mkdir -p a/bb/ccc<br>
> $ cpio -i <a.cpio<br>
> cpio: a/bb not created: newer or same age version exists<br>
> 1 block<br>
> $ touch -d yesterday a/bb<br>
> $ cpio -i <a.cpio<br>
> cpio: cannot remove current a/bb: Directory not empty<br>
> 1 block<br>
> $ cpio -iu <a.cpio<br>
> cpio: cannot remove current a/bb: Directory not empty<br>
> 1 block<br>
> $ # Never replaces an existing dir<br>
<br>
Does having a directory where you want to put a file come up often?<br>
<br>
What exactly is the advantage of the special case of removing an empty directory<br>
where you want to put a file? Is this a common case? Is there a use case for it?<br>
Is there any design idea here other than "match whatever gnu does"?<br></blockquote><div><br></div><div>As far as I can tell not often and I don't know why GNU cpio does this either.</div><div>My goal was to make "file replace file" and "dir replace dir" work. (ah right I forgot to add the "dir replace dir" testcase, but one can also argue that directory is just a special type of file...)</div><div>"dir replace file" and "file replace dir" are artifacts of the way I implemented the patch, because I unlink/rmdir a name regardless of the file type I'm going to create later.</div><div>I wasn't trying to match GNU's behavior, it's just that my impl happens to behave like GNU's... </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Is this something tar does? Sigh. Debian's tar -x is replacing a directory with<br>
a file, but toybox's isn't. And my tar.c is dirty because of pending tar<br>
--transform stuff... right, one more todo item.<br>
<br>
> On Sat, Feb 27, 2021 at 3:03 PM Rob Landley <<a href="mailto:rob@landley.net" target="_blank" class="cremed">rob@landley.net</a><br>
> <mailto:<a href="mailto:rob@landley.net" target="_blank" class="cremed">rob@landley.net</a>>> wrote:<br>
> <br>
> On 2/20/21 8:34 AM, Yi-yo Chiang via Toybox wrote:<br>
> > *ping* (in case this got categorized as spam by gmail)<br>
> <br>
> It did, and so did this reply. :)<br>
> <br>
> > On Sun, Feb 14, 2021 at 9:13 PM Yi-Yo Chiang <<a href="mailto:yochiang@google.com" target="_blank" class="cremed">yochiang@google.com</a><br>
> <mailto:<a href="mailto:yochiang@google.com" target="_blank" class="cremed">yochiang@google.com</a>><br>
> > <mailto:<a href="mailto:yochiang@google.com" target="_blank" class="cremed">yochiang@google.com</a> <mailto:<a href="mailto:yochiang@google.com" target="_blank" class="cremed">yochiang@google.com</a>>>> wrote:<br>
> ><br>
> > If -u is specified, then replace any existing files or directories.<br>
> <br>
> When does it replace directories? Do we replace a directory with a file, or a<br>
> file with a directory? (What if the directory to be replaced with a file isn't<br>
> empty, ala rm -r vs rmdir?)<br>
> <br>
> <br>
> Honestly I'm not sure, and IIUC the spec doesn't say what to do either<br>
> (<a href="https://pubs.opengroup.org/onlinepubs/7908799/xcu/cpio.html" rel="noreferrer" target="_blank" class="cremed">https://pubs.opengroup.org/onlinepubs/7908799/xcu/cpio.html</a>), so I assumed it<br>
> is implementation specific?<br>
> In this case I choose the behavior that is simplest to implement (IMO).<br>
<br>
Replacing an existing file with another file makes sense. Replacing a file with<br>
a directory makes a certain amount of sense because there's a common failure<br>
mode: "cp file /usr/bin" will make a file if bin doesn't exist.<br>
<br>
It's replacing a directory with a file that makes me go "huh"? For one thing<br>
"rmdir" is not nearly as commonly used as "rm -r", and mkdir -p path/to/file<br>
does not let you rmdir "path/to" (it's not empty, the single command is a<br>
multi-command cleanup, unless you use rm -r)...<br>
<br>
*shrug* If it's what the other one does we can do the same thing in the absence<br>
of a spec.<br>
<br>
> * If the *path* to be extracted already existed, then unlink/rmdir the path. *<br>
<br>
That's the problem: you can't rmdir a path you can only rmdir a single empty<br>
directory. and supporting an empty directory is like supporting a zero length<br>
file but failing if the file has any contents.<br>
<br>
> So yes we replace a file with a directory, and replace a directory (so long as<br>
> rmdir() success, which means directory must be empty) with a file.<br>
<br>
It's the "must be empty" part that strikes me as weird. Of course if we let it<br>
rm -r stuff the command immediately becomes more _dangerous_, which is probably<br>
why they don't do it. But extracting over something that disagrees whether files<br>
or directories should be there is also kinda weird...<br></blockquote><div><br></div><div><div>Allow me to retcon:</div><div> If the *name* to be extracted already existed, then *unlink/rmdir* the name.</div><div><br></div><div>I think we are taking different mindsets here. I don't actually care if the name to be removed is empty or not, file or directory.</div><div>My main concern is whether I can remove "name" before I extract "name".</div></div><div>If rmdir or unlink failed, then I'll just bailout.</div><div>"name must be empty dir" is just a requirement for rmdir to succeed.</div><div>And yes not doing "rm -rf" because it sounds like fire.</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
In the absence of a spec or a consistent design idea, I tend to look at what tar<br>
and zip do and see if there's consistency there, and there sort of is but I need<br>
to go fix tar, and I never got to implement zip because stunlocked with other<br>
todo items, and I was trying to do sh.c but have been too busy to commit<br>
anything to that in a month, and I'm so very tired...<br>
<br>
> Let's see...<br>
> <br>
> $ mkdir walrus<br>
> $ echo hello > walrus/file<br>
> $ echo walrus/file | cpio -H newc -o > blah.cpio<br>
> $ cpio -H newc -u -i < blah.cpio<br>
> 1 block<br>
> $ mv walrus walrus2<br>
> $ touch walrus<br>
> $ cpio -H newc -i < blah.cpio<br>
> cpio: walrus/file: Cannot open: Not a directory<br>
> 1 block<br>
> $ cpio -H newc -u -i < blah.cpio<br>
> cpio: walrus/file: Cannot open: Not a directory<br>
> 1 block<br>
> $ cpio -H newc -dui < blah.cpio<br>
> cpio: `walrus' exists but is not a directory<br>
> cpio: walrus/file: Cannot open: Not a directory<br>
> 1 block<br>
> <br>
> Nope, doesn't look like it. Let's see, will it replace a /dev node...? Yes it<br>
> does (rather than writing the contents to it. What's the default tar behavior<br>
> here... replaces the dev node. Which is not what solaris did back in college, I<br>
> note. Why do we want cpio's default behavior to differ from tar's here?)<br>
> <br>
> I didn't follow.. Are you saying GNU tar replaces a device node, and Solaris tar<br>
> writes to a device node?<br>
<br>
Circa 1994 it would open whatever was there at that path, truncate it, and write<br>
to it. Truncate was ignored on a block device, so... (It was used as a way to<br>
image floppies.)<br>
<br>
The current one doesn't do that, so I guess that's the expected behavior now,<br>
which raises the question of why cpio needs -u when -u is the default for tar<br>
(and doing -u on "older" files is the default for cpio, which is another head<br>
scratcher. Seems kinda fancy given the userbase of this command...)<br></blockquote><div><br></div><div>I agree it's a strange thing to do, and that's why I didn't check any timestamp, I either replace all name (-u) or skip any existing name (without -u). </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
This is similar to how the initial initramfs cpio extract code would open the<br>
existing node (in O_APPEND mode instead of O_TRUNC mode) and write data to it.<br>
Not what the current one does, but I'm trying to work out the design reasons for<br>
what it's doing. (Alas posix deleted the cpio spec, presumably because RPM and<br>
initramfs are based on it and therefore Jorg Schilling felt it needed to die<br>
because Linux cooties.)<br>
<br>
> Do we want toybox cpio/tar to be feature-identical to GNU or Solaris'<br>
> implementation? (I hope not, cause that feels like a bloat to code size)<br>
<br>
I'm trying to figure out what the right thing to do is here. My first instinct<br>
is "if tar and zip behave the same way, cpio probably should too", but I don't<br>
want to break somebody's script.<br>
<br>
What's the use case that motivated these changes again?<br>
<br>
> In any case, it looks like the expected behavior is to open with O_EXCL and<br>
> unlink if it failed. Under what circumstances DOES it replace an existing file<br>
> with a directory, or an existing directory with a file? (The tests you submitted<br>
> only replace files with other files.)<br>
> <br>
> See my example at the start.<br>
> Your commands didn't replace "walrus" because your archive extracts<br>
> "walrus/file" but not "walrus".<br>
<br>
I know why it didn't, yes.<br>
<br>
> If you use "find ... | cpio -o" to create the test archive, then the existing<br>
> "walrus" would be replaced by cpio when cpio is trying to mkdir(walrus).<br>
<br>
Because there would then be _two_ entries in the cpio archive, and the directory<br>
entry is guaranteed to come before the directory contents by the definition of<br>
the find -depth option, which is a different test than the one I was running.<br>
<br>
But if you tell cpio -d it creates leading directories where needed, and in<br>
doing so it did _not_ remove conflicting files. And I don't understand why -d<br>
ignores -u, other than it being an oversight. (That's why the last test I ran<br>
combined -d and -u. Exact matches are treated differently than path components.)<br>
<br>
By the way, I suspect that zapping directories was a side effect of zapping<br>
symlinks, as an attack mitigation strategy. I had to care about this when<br>
implementing tar, but can't find the test case or blog entry. It was while I was<br>
working at Johnson Controls, I want to say early 2019? Hmmm, it wasn't<br>
<a href="https://nvd.nist.gov/vuln/detail/CVE-2016-6321" rel="noreferrer" target="_blank" class="cremed">https://nvd.nist.gov/vuln/detail/CVE-2016-6321</a> and I remember it was something<br>
busybox got wrong at the time...<br>
<br>
Oh great, half of tests/tar.test is just notes-to-self not actual tests. So much<br>
half-finished stuff I've never had time to finish. Well, can't do it right now...<br>
<br>
> I'd like to avoid the separate "stat" if necessary not just because it's extra<br>
> code but because it's a race condition between two non-atomic operations waiting<br>
> to happen. (I have no idea if the gap is exploitable, but I don't want to worry<br>
> about it if just not doing that is an option? And not doing that would be open<br>
> O_EXCL and on failure unlink and O_EXCL a second time.)<br>
> <br>
> Yeah I totally agree. But I'd like to point out that in my first iteration of<br>
> this code, I do guard the stat() with various conditions and nested if-else-ifs,<br>
<br>
If all we care about is a single attempt to recover an occluded exact match the<br>
logic sounds like:<br>
<br>
i = 0;<br>
do if (-1 != (fd = open(O_CREAT|O_EXCL))) break;<br>
while (!i++ && (!unlink(name) || !rmdir(name));<br>
if (fd == -1) {<br>
perror_msg_raw(name);<br>
continue;<br>
}<br>
<br>
Where does the stat come in?<br>
<br>
(Ok, if you chmod 000 a normal file the error message printed would be ENOTDIR<br>
from the failing rmdir(), technically it should fall back to the rmdir only if<br>
the unlink returns EISDIR, which the linux man page says is a non-posix value<br>
returned by Linux but the BSD man page _also_ says is what unlink(2) returns so<br>
I'm assuming mac does too.)<br>
<br>
> hoping to only call stat() if EEXIST error happened.<br>
> However the code quickly grew too complex and unreadable, with too many<br>
> combinations to consider (file replace dir, dir replace file, dir replace dir,<br>
> what if dir not empty? what about symlink and special files...), so I changed<br>
> the implementation to the one you're seeing now.<br>
> This is also the reason why I add the "if dir already exist, then just do chmod"<br>
> shortcut, because race could happen between rmdir(a) and mkdir(a). <br>
<br>
A race resulting in a failure, you mean?<br></blockquote><div><br></div><div>I mean another process may create "a" between rmdir(a) and mkdir(a).</div><div>Second thoughts, this is not a real issue because that would just make mkdir(a) to fail with EEXIST...</div><div>I should just say to save an extra syscall. (rmdir(a) + mkdir(a) vs chmod(a, ...))</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
$ ln -s broken nowhere<br>
$ ls -l nowhere<br>
lrwxrwxrwx 1 landley landley 6 Feb 28 03:09 nowhere -> broken<br>
$ mkdir nowhere<br>
mkdir: cannot create directory ‘nowhere’: File exists<br>
<br>
How is this exploitable?<br>
<br>
The reason I'm spending time on this thread instead of staring at the cpio code<br>
is that implementation is easy, figuring out what it SHOULD do is hard. I still<br>
don't know what use case motivated your changes. Your patch description says<br>
what but not why, I have to reverse engineer an understanding of what you're<br>
trying to accomplish from the conclusion you came to about the best way to<br>
implement it.<br>
<br>
> Another point is that when replacing an existing file, having a separate stat()<br>
> would mean:<br>
> stat(a) & unlink(a) & open(a, O_EXCL)<br>
> but not stat()-ing beforehand would mean:<br>
> open(a, O_EXCL) [fail] & stat(a) & ("a" is a directory ? rmdir : unlink)(a)<br>
> & open(a, O_EXCL)<br>
<br>
Isn't the open(a, O_EXCL) succeeding the common case though? And if that's the<br>
first thing it tries and it succeeds...<br>
<br>
> So having a separate "stat" doesn't necessarily mean an extra system call.<br>
> If path doesn't exist then the stat() is an extra syscall, otherwise the stat()<br>
> saves an extra syscall. <br>
<br>
Mostly it's just the possibility that the thing I statted is not the thing I<br>
acted upon a moment later making me shy away from it as a first choice for<br>
implementation. I prefer not to have to reason about what _could_ happen in that<br>
gap if there's another way (such as the open saying it didn't work and then<br>
unlink saying it didn't work so you have to rmdir; the first common case didn't<br>
succeed and the second common case didn't succeed so then try the third. Does a<br>
failed unlink attempt take longer than a stat?)<br></blockquote><div><br></div><div>Both work for me. I simply chose a different implementation. I don't know the overhead of stat() / unlink() though... I always assumed them to be amortized insignificant due to the file system caching the inode?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
I note that the obvious approach is to wrap the whole if/else staircase with a<br>
retry loop (otherwise there's a goto), which makes the patch look big because of<br>
the extra indentation layer (which is why git diff supports -b, and yes I've<br>
done that on patches to see what people actually did...)<br>
<br>
> (You'll notice the existing stat was an fstat() on an already open file<br>
> descriptor, and you're adding an lstat on a path which could get swapped out<br>
> afterwards. In general I try not to do that because it seems racy with programs<br>
> dropping symlinks based on inotify triggers. Yeah you've sprayed the system down<br>
> with selinux but I don't want to worry about it at a design level...)<br>
> <br>
> Unfortunately I haven't had the spare focus this week to properly wrap my brain<br>
> around what I want to happen here. I finally got a chance to look at it the end<br>
> of the day friday, and just wanted to say why I haven't applied it as-is yet.<br>
> I'll try to redo it this weekend, but first I need to confirm that open(O_EXCL)<br>
> is sufficient error detection and what we do about conflicting directory<br>
> creation (is there a case where -u DOES apply to -d?) <br>
> <br>
> I hope not, let's keep it simple...<br>
<br>
It looks like only exact matches of empty directories get rmdir()ed (and really<br>
it's unlinkat(AT_REMOVEDIR) and why can unlink remove files, dev nodes,<br>
symlinks, sockets, and fifos, but directories have their own remove call (even<br>
though that call fails if the directory isn't empty)? Why is that again? I'm too<br>
tired for this to make sense right now...<br>
<br>
> and adding tests for the<br>
> corner cases limning in the design decisions.<br>
> <br>
> I can add more tests once we decide the desired behavior.<br>
Tests are fiddly. I really HATE when TEST_HOST gives spurious irreproducible<br>
failures:<br>
<br>
FAIL: tar sparse without overflow<br>
echo -ne '' | tar c --owner root --group root --mtime @1234567890 --sparse fweep<br>
| SUM 3<br>
--- expected 2021-02-28 09:28:39.518369796 +0000<br>
+++ actual 2021-02-28 09:28:39.526369797 +0000<br>
@@ -1 +1 @@<br>
-e1560110293247934493626d564c8f03c357cec5<br>
+ce3f8e6b10723ce4d85f06c606bc634a9db23be0<br>
.singlemake:1781: recipe for target 'test_tar' failed<br>
make: *** [test_tar] Error 1<br>
<br>
Smells like maybe clock edge overflow of a second ticking over and winding up<br>
altering a header field? But I didn't grab the artifacts out of generated/ to<br>
see what it was and it refuses to do it again...<br>
<br>
Anyway, added a new test to tar and made it unlink an empty directory where it<br>
wants to put a file. If the result of all this is "cpio should work like tar<br>
when files/directories conflict with archive contents" then that's easy to<br>
implement. If not, I'd like to understand _why_ not.<br>
<br>
> I need to properly wrap my head around this, but haven't been at my best the<br>
> past couple weeks. Slightly overstretched. Sorry 'bout that.<br>
> <br>
> Nah it's cool. Do take care and take your time and don't feel burned out. <br>
<br>
Too late, but it's not because of this.<br>
<br>
I _should_ just sit down and work out what to do and implement it instead of<br>
having email threads that boil down to "I haven't scraped up enough brain to<br>
work through all these corner cases myself yet", but I've been tired and<br>
headachey for a few days now. I'd blame cedar pollen but my wife (in<br>
minneapolis) has been nauseous for 2 days and apparently it's going around?<br>
<br>
<a href="https://twitter.com/catacalypto/status/1365538065184251905" rel="noreferrer" target="_blank" class="cremed">https://twitter.com/catacalypto/status/1365538065184251905</a><br>
<br>
To be honest I expect I'm still limp from the blizzard last week, which I<br>
_shouldn't_ be but 2020 kinda burned through my reserves. Just because the<br>
people who tried to violently overthrow the government last month have<br>
deregulated infrastructure in my state to the point civilization literally<br>
collapsed around here last week, and yesterday a jewish friend broke down<br>
sobbing because the CPAC conference's stage is literally shaped like a nazi<br>
symbol...<br>
<br>
But there's good news: my house _didn't_ get flooded this time (after the first<br>
4 times my wife has outright PTSD about that but she's been getting her<br>
doctorate in another state for years and we haven't been able to see each other<br>
in person since the pandemic started). The boil water advisory here was lifted<br>
all the way back on wednesday, our power company isn't one of the ones charging<br>
5 figure electricity bills (instead they say they'll spread it out over the next<br>
10 years), and as of yesterday the local grocery store has restocked over half<br>
its shelves. Plus the Johnson and Johnson one dose vaccine got approved, which<br>
is the one I've been rooting for since brexit derailed availability of the<br>
oxford one. If everything goes well I'm hoping I can leave the house and go<br>
inside another building (other than the grocery store down the street) by maybe<br>
september. (I still have the phobia of needles resulting from a nurse wheeling<br>
in a tray of 24 needles for allergy testing when I was 7 and tying me up in a<br>
sheet when I freaked out about it until I threw up on her. Ever since I've been<br>
able to make my gums bleed by thinking to hard about needles, but I can usually<br>
psych myself up to get an injection without losing consciousness given about<br>
half an hour. So I have that to look forward to, but this is part of the reason<br>
I really want the one dose version instead of the two dose version...)<br>
<br>
Anyway, weekend! In theory this means spare cycles to throw at the toybox pull<br>
request and bug report backlog (instead of working on the shell I've been trying<br>
to finish, but nobody except me uses that and other people use the stuff they're<br>
sending me patches for, so...)<br>
<br>
Rob<br>
<br>
P.S. Sorry I haven't been more focused. And Pascal's Apology about the length of<br>
the email:<br>
<a href="https://www.npr.org/sections/13.7/2014/02/03/270680304/this-could-have-been-shorter" rel="noreferrer" target="_blank" class="cremed">https://www.npr.org/sections/13.7/2014/02/03/270680304/this-could-have-been-shorter</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><table width="90%" border="0" cellspacing="0" cellpadding="0" style="margin:0px;padding:0px;font-family:"Times New Roman";max-width:348px"><tbody style="margin:0px;padding:0px"><tr style="margin:0px;padding:0px"><td style="padding:0px"><table border="0" cellspacing="0" cellpadding="0" style="margin:0px;padding:20px 0px 0px"><tbody style="margin:0px;padding:0px"><tr style="margin:0px;padding:0px"><td valign="top" style="padding:0px 20px 0px 0px;vertical-align:top;border-right:1px solid rgb(213,213,213)"><img src="https://i.imgur.com/eGpkLls.png" width="200" height="64"><br></td><td style="padding:0px 0px 0px 20px"><table border="0" cellspacing="0" cellpadding="0" style="margin:0px;padding:0px"><tbody style="margin:0px;padding:0px"><tr style="margin:0px;padding:0px"><td colspan="2" style="font-family:Arial,Helvetica,Verdana,sans-serif;padding:1px 0px 5px;font-size:13px;line-height:13px;color:rgb(56,58,53);font-weight:700">Yi-yo Chiang</td></tr><tr style="margin:0px;padding:0px"><td colspan="2" style="font-family:Arial,Helvetica,Verdana,sans-serif;padding:0px 0px 5px;font-size:11px;line-height:13px;color:rgb(56,58,53)">Software Engineer</td></tr><tr style="margin:0px;padding:0px"><td colspan="2" style="font-family:Arial,Helvetica,Verdana,sans-serif;padding:0px 0px 5px;font-size:11px;line-height:13px;color:rgb(56,58,53)"><a href="mailto:yochiang@google.com" target="_blank" class="cremed">yochiang@google.com</a></td></tr><tr style="margin:0px;padding:0px"><td colspan="2" style="font-family:Arial,Helvetica,Verdana,sans-serif;padding:0px 0px 3px;font-size:11px;line-height:13px;color:rgb(3,112,248)"></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table></div></div></div></div>