[Toybox] realpath/tar status

enh enh at google.com
Mon Feb 6 10:07:49 PST 2023 

On Fri, Feb 3, 2023 at 10:29 PM Rob Landley <rob at landley.net> wrote:
>
>
>
> On 2/3/23 17:29, enh via Toybox wrote:
> > (sorry for the brevity, but i'm running out of day without having had
> > any time to spend on this, so i'd rather say _something_ than wait
> > until i have more...)
> >
> > feedback on realpath:
> > """
> > ./prebuilts/build-tools/path/linux-x86/realpath
> > /mnt/sdc/android/kernel/out --relative-to /mnt/sdc/android/kernel
> >
> > For some reason it returns /out not out, which looks incorrect. Am I
> > using it incorrectly?
> > """
>
> That is a bug, thanks. Reproduced...
>
> > -*-
> >
> > as for the new tar, i updated the prebuilts yesterday and we've seen
> > enough OOM kills since that i've had to revert it.
>
> Grrr. I've got a couple of suspects for what I screwed up, but... Lemme see if I
> can work the ASAN leak detector into my workflow at all here. (There's a lot of
> commands that intentionally leak resources on the way out because the OS will
> free them, but there's a category of commands that should NOT do that because
> they process theoretically unbounded input and cannot be allowed to leak-per.)
>
> > every case i saw
> > was from https://cs.android.com/android/platform/superproject/+/master:system/update_engine/Android.bp;l=947?q=ue_unittest_disk_imgs
> >
> > so basically:
> > ```
> > tar -jxf sample_images/sample_images.tar.bz2 -C gen disk_ext2_1k.img
> > disk_ext2_4k.img disk_ext2_4k_empty.img disk_ext2_unittest.img
> > ```
> >
> > (direct link to the .tar.bz2:
> > https://android.googlesource.com/platform/system/update_engine/+/refs/heads/master/sample_images/)
>
> Wait, OOM on _extract_? Did I change the extract codepath? (That's sounding more
> like I damaged dirtree in the non-breadth case...)

yeah, that didn't make much sense to me either, and i should have
explicitly said "i'm seeing tar killed by SIGKILL and _assume_ that's
the oom killer". but -- now i've had time to repro -- it looks like
suicide instead:

~/aosp-master-with-phones$ ./out/host/linux-x86/bin/toybox tar -jxf
system/update_engine/sample_images/sample_images.tar.bz2 -C
out/soong/.temp/sbox/c744f840647dbf476d005753d9a7965d5af2771b/out/gen
disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img
disk_ext2_unittest.img
tar: had errors

...

~/aosp-master-with-phones$ strace ./out/host/linux-x86/bin/toybox tar
-jxf system/update_engine/sample_images/sample_images.tar.bz2 -C
out/soong/.temp/sbox/c744f840647dbf476d005753d9a7965d5af2771b/out/gen
disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img
disk_ext2_unittest.img
...
lseek(3, 2097152, SEEK_SET)             = 2097152
read(4, "PAYLOAD_MINOR_VERSION=1234\n\0\0\0\0\0"..., 4096) = 4096
write(3, "PAYLOAD_MINOR_VERSION=1234\n\0\0\0\0\0"..., 4096) = 4096
read(4, "CHROMEOS_AUSERVER=https://tools."..., 4096) = 4096
write(3, "CHROMEOS_AUSERVER=https://tools."..., 4096) = 4096
read(4, "#!/etc/../bin/sh\necho \"I'm a pos"..., 4096) = 4096
write(3, "#!/etc/../bin/sh\necho \"I'm a pos"..., 4096) = 4096
read(4, "#!/etc/../bin/sh\nexit 3\n\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(3, "#!/etc/../bin/sh\nexit 3\n\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(4, "#!/etc/../bin/sh\nexit 1\n\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(3, "#!/etc/../bin/sh\nexit 1\n\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(4, "#!/etc/../bin/sh\ntrap \"{ echo Go"..., 4096) = 4096
write(3, "#!/etc/../bin/sh\ntrap \"{ echo Go"..., 4096) = 4096
read(4, "#!/etc/../bin/sh\n# These values "..., 4096) = 4096
write(3, "#!/etc/../bin/sh\n# These values "..., 4096) = 4096
read(4, "#!/etc/../bin/sh\necho \"This is m"..., 4096) = 4096
write(3, "#!/etc/../bin/sh\necho \"This is m"..., 4096) = 4096
read(4, "#!/etc/../bin/sh\necho \"This is m"..., 4096) = 4096
write(3, "#!/etc/../bin/sh\necho \"This is m"..., 4096) = 4096
lseek(3, 4194304, SEEK_SET)             = 4194304
ftruncate(3, 4194304)                   = 0
close(3)                                = 0
geteuid()                               = 73769
chmod("disk_ext2_unittest.img", 0600)   = 0
utimensat(AT_FDCWD, "disk_ext2_unittest.img", [{tv_sec=1616696299,
tv_nsec=0} /* 2021-03-25T18:18:19+0000 */, {tv_sec=1616696299,
tv_nsec=0} /* 2021-03-25T18:18:19+0000 */], AT_SYMLINK_NOFOLLOW) = 0
read(4, "disk_sqfs_empty.img\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
lseek(4, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
read(4, "hsqs\1\0\0\0\353\323\\`\0\0\2\0\0\0\0\0\5\0\21\0\300\4\1\0\4\0\0\0"...,
4096) = 4096
read(4, "disk_sqfs_default.img\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
lseek(4, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
read(4, "hsqs\25\0\0\0\354\323\\`\0\0\2\0\1\0\0\0\5\0\21\0\300\4\1\0\4\0\0\0"...,
4096) = 4096
read(4, "\37\0\1\0\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"...,
4096) = 4096
read(4, "\237foo\nbar\na\1\0\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"...,
4096) = 4096
read(4, "disk_sqfs_unittest.img\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
lseek(4, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
read(4, "hsqs\16\0\0\0\354\323\\`\0\0\2\0\1\0\0\0\5\0\21\0\300\4\1\0\4\0\0\0"...,
4096) = 4096
read(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
512) = 512
read(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
512) = 512
kill(3992265, SIGKILL)                  = 0
wait4(3992265, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 3992265
exit_group(0)                           = ?
+++ exited with 0 +++

(yes, that pid is legit. now you see why i needed the column size fix
for ps/top :-) )

(note that the command line there should give you convenient repro
steps --- you don't need a whole AOSP tree, just the .tar.bz2 i linked
to in the first post.)

> > the input is pretty small:
> > ```
> > ~/aosp-master-with-phones/system/update_engine$ ls -lh
> > sample_images/sample_images.tar.bz2
> > -rw-r----- 1 enh primarygroup 5.9K Nov 14 18:51
> > sample_images/sample_images.tar.bz2
> > ```
> > and the outputs are only 4MiB too, so i'm not sure what's going on
> > there... (and we haven't updated bzip2 since 1.0.8 in 2021, so --
> > although, yes, all the build tool prebuilts are updated as one -- that
> > didn't change, only toybox did.)
>
> Thanks. On it. I'll try to get you fixes by monday.
>
> Hmmm, I kind of need a "create/extract kernel tarball" sort of test, but it's
> awkward to stick that into "make tests"...
>
> Rob

More information about the Toybox mailing list