From rob at landley.net Thu Dec 13 08:01:17 2012 From: rob at landley.net (Rob Landley) Date: Thu, 13 Dec 2012 02:01:17 -0600 Subject: [Aboriginal] Working on next release. Message-ID: <1355385677.18402.0@driftwood> Since the 3.7 kernel dropped monday evening I've been banging away at it. Updating arm patches, perl removal patches, and so on. They've got a special init data section that drives gcc 4.2 nuts (build break due to section mismatch), which they've got a config symbol for but only parisc is using it. I'd like to try to figure out how to patch the toolchain I'm using to deal with this, but for right now I've just made that config symbol default to Y. (Downloaded the git repo thinking I could bisect it and see what they did to fix it, but there are no labels for releases anywhere in the tree, nor comments about releases in the commits. It's converted from svn, I think? Ginormous mess, and building it requires mpfr and such... Not something I'm likely to deal with today.) I vaguely plan to post the updated perl removal patches to linux-kernel once I've done a bit more testing. I also need to cut a toybox release because I'm testing with the current hg snapshot of toybox. The plan is still to switch to musl, but the kernel release happened before I was ready and I don't want to fall behind again. Rob 1355385677.0 From bjorn.forsman at gmail.com Wed Dec 26 20:55:04 2012 From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=) Date: Wed, 26 Dec 2012 21:55:04 +0100 Subject: [Aboriginal] Merry christmas, I have found two bugs Message-ID: Hi, Since it's christmas and I have som extra time, I thought I'd play a bit with aboriginal :-) And now I think I've found two bugs. Bug 1 is a segmentation fault during system image boot: $ cd build/system-image-avmv5l $ ./run-emulator.sh [...] Freeing init memory: 96K Segmentation fault 8139cp 0000:00:0c.0 eth0: link up, 100Mbps, full-duplex, lpa 0x05E1 Not using distcc. Type exit when done. (armv5l:1) /home # Not sure what exactly is segfaulting, but segfault == bug to me. And it only happens with ./run-emulator.sh, not ./dev-environment.sh. And bug 2 is a floating point exception in toybox "ls": $ ./dev-environment.sh [...] Freeing init memory: 96K 8139cp 0000:00:0c.0 eth0: link up, 100Mbps, full-duplex, lpa 0x05E1 EXT4-fs (sdb): mounting ext3 file system using the ext4 subsystem EXT4-fs (sdb): recovery complete EXT4-fs (sdb): mounted filesystem with ordered data mode. Opts: (null) Distcc acceleration enabled. Type exit when done. (armv5l:1) /home # ls Floating point exception (armv5l:1) /home # toybox ls Floating point exception (armv5l:1) /home # ls -l total 24 drwx------ 2 root root 12288 2012-12-26 20:39 lost+found (armv5l:1) /home # The really strange thing (at least to me) is that the floating point exception only appears in ./dev-environment.sh, NOT ./run-emulator.sh. And it's only "ls" that is broken, "ls -l" is ok. Best regards, Bj?rn Forsman 1356555304.0 From rob at landley.net Wed Dec 26 21:36:35 2012 From: rob at landley.net (Rob Landley) Date: Wed, 26 Dec 2012 15:36:35 -0600 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: (from bjorn.forsman@gmail.com on Wed Dec 26 14:55:04 2012) Message-ID: <1356557795.31232.5@driftwood> On 12/26/2012 02:55:04 PM, Bj?rn Forsman wrote: > Hi, > > Since it's christmas and I have som extra time, I thought I'd play a > bit with aboriginal :-) Yay! Glad to have you. > And now I think I've found two bugs. Bug 1 is > a segmentation fault during system image boot: > > $ cd build/system-image-avmv5l > $ ./run-emulator.sh > [...] > Freeing init memory: 96K > Segmentation fault > 8139cp 0000:00:0c.0 eth0: link up, 100Mbps, full-duplex, lpa 0x05E1 > Not using distcc. > Type exit when done. > (armv5l:1) /home # > > Not sure what exactly is segfaulting, but segfault == bug to me. And > it only happens with ./run-emulator.sh, not ./dev-environment.sh. There are somewhat different code paths in the init.sh shell script so it's probably something only running in the first case. I need to rebuild the armv5l target to test this, that'll take a few minutes... > And bug 2 is a floating point exception in toybox "ls": > $ ./dev-environment.sh > [...] > Freeing init memory: 96K > 8139cp 0000:00:0c.0 eth0: link up, 100Mbps, full-duplex, lpa 0x05E1 > EXT4-fs (sdb): mounting ext3 file system using the ext4 subsystem > EXT4-fs (sdb): recovery complete > EXT4-fs (sdb): mounted filesystem with ordered data mode. Opts: (null) > Distcc acceleration enabled. > Type exit when done. > (armv5l:1) /home # ls > Floating point exception > (armv5l:1) /home # toybox ls > Floating point exception > (armv5l:1) /home # ls -l > total 24 > drwx------ 2 root root 12288 2012-12-26 20:39 lost+found > (armv5l:1) /home # *boggle* That's... very strange. (Also, I didn't think ls was using floating point? Quick check says it's not...) > The really strange thing (at least to me) is that the floating point > exception only appears in ./dev-environment.sh, NOT ./run-emulator.sh. > And it's only "ls" that is broken, "ls -l" is ok. Very wierd. Lemme see if I can reproduce that... yes I can. And wow, you're right, it's only happening for dev-environment.sh which should be TOTALLY unrelated... wget http://landley.net/aboriginal/strace-armv5l chmod +x strace-armv5l ./strace /bin/ls execve("/bin/ls", ["/bin/ls"], [/* 9 vars */]) = 0 brk(0) = 0x932000 brk(0x9324b0) = 0x9324b0 set_tls(0x932490, 0x5124c, 0, 0x1, 0x65fac) = 0 ioctl(0, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 getuid32() = 0 geteuid32() = 0 brk(0x9334b0) = 0x9334b0 brk(0x934000) = 0x934000 umask(0) = 022 umask(022) = 0 ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(0, TIOCGWINSZ, {ws_row=0, ws_col=0, ws_xpixel=0, ws_ypixel=0}) = 0 ioctl(1, TIOCGWINSZ, {ws_row=0, ws_col=0, ws_xpixel=0, ws_ypixel=0}) = 0 ioctl(2, TIOCGWINSZ, {ws_row=0, ws_col=0, ws_xpixel=0, ws_ypixel=0}) = 0 newfstatat(AT_FDCWD, ".", {st_mode=S_IFDIR|0755, st_size=1024, ...}, 0) = 0 open(".", O_RDONLY|O_LARGEFILE) = 3 dup(3) = 4 fstat64(4, {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0 fcntl64(4, F_GETFL) = 0x20000 (flags O_RDONLY|O_LARGEFILE) getdents64(4, /* 4 entries */, 1024) = 120 newfstatat(4, ".", {st_mode=S_IFDIR|0755, st_size=1024, ...}, AT_SYMLINK_NOFOLLOW) = 0 newfstatat(4, "strace-armv5l", {st_mode=S_IFREG|0755, st_size=329448, ...}, AT_SYMLINK_NOFOLLOW) = 0 newfstatat(4, "..", {st_mode=S_IFDIR|0755, st_size=163, ...}, AT_SYMLINK_NOFOLLOW) = 0 newfstatat(4, "lost+found", {st_mode=S_IFDIR|0700, st_size=12288, ...}, AT_SYMLINK_NOFOLLOW) = 0 getdents64(4, /* 0 entries */, 1024) = 0 close(4) = 0 gettid() = 49 tgkill(49, 49, SIGFPE) = 0 --- SIGFPE {si_signo=SIGFPE, si_code=SI_TKILL, si_pid=49, si_uid=0} --- +++ killed by SIGFPE +++ Floating point exception So it's getting a ways into ls. Looks like it manages the whole dirtree and then dies on the way back. Hmmm... Thanks for the heads up, I'm going to chew on these for a bit... Rob 1356557795.0 From bjorn.forsman at gmail.com Wed Dec 26 22:23:06 2012 From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=) Date: Wed, 26 Dec 2012 23:23:06 +0100 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: <1356557795.31232.5@driftwood> References: <1356557795.31232.5@driftwood> Message-ID: On 26 December 2012 22:36, Rob Landley wrote: > On 12/26/2012 02:55:04 PM, Bj?rn Forsman wrote: [...] >> And now I think I've found two bugs. Bug 1 is >> a segmentation fault during system image boot: >> >> $ cd build/system-image-avmv5l >> $ ./run-emulator.sh >> [...] >> Freeing init memory: 96K >> Segmentation fault >> 8139cp 0000:00:0c.0 eth0: link up, 100Mbps, full-duplex, lpa 0x05E1 >> Not using distcc. >> Type exit when done. >> (armv5l:1) /home # >> >> Not sure what exactly is segfaulting, but segfault == bug to me. And >> it only happens with ./run-emulator.sh, not ./dev-environment.sh. > > There are somewhat different code paths in the init.sh shell script so it's > probably something only running in the first case. I need to rebuild the > armv5l target to test this, that'll take a few minutes... > >> And bug 2 is a floating point exception in toybox "ls": >> $ ./dev-environment.sh >> [...] >> Freeing init memory: 96K >> 8139cp 0000:00:0c.0 eth0: link up, 100Mbps, full-duplex, lpa 0x05E1 >> EXT4-fs (sdb): mounting ext3 file system using the ext4 subsystem >> EXT4-fs (sdb): recovery complete >> EXT4-fs (sdb): mounted filesystem with ordered data mode. Opts: (null) >> Distcc acceleration enabled. >> Type exit when done. >> (armv5l:1) /home # ls >> Floating point exception >> (armv5l:1) /home # toybox ls >> Floating point exception >> (armv5l:1) /home # ls -l >> total 24 >> drwx------ 2 root root 12288 2012-12-26 20:39 lost+found >> (armv5l:1) /home # > > *boggle* That's... very strange. > > (Also, I didn't think ls was using floating point? Quick check says it's > not...) Neither did I. But wait... I just read up on SIGFPE and it's not necessarily a floating point error, it's an arithmetic error. For example, division by zero yields SIGFPE (just checked it). >> The really strange thing (at least to me) is that the floating point >> exception only appears in ./dev-environment.sh, NOT ./run-emulator.sh. >> And it's only "ls" that is broken, "ls -l" is ok. > > Very wierd. Lemme see if I can reproduce that... yes I can. And wow, you're > right, it's only happening for dev-environment.sh which should be TOTALLY > unrelated... > > wget http://landley.net/aboriginal/strace-armv5l That returns 404 error code for me (but ignore that). > chmod +x strace-armv5l > ./strace /bin/ls > > execve("/bin/ls", ["/bin/ls"], [/* 9 vars */]) = 0 > brk(0) = 0x932000 > brk(0x9324b0) = 0x9324b0 > set_tls(0x932490, 0x5124c, 0, 0x1, 0x65fac) = 0 > ioctl(0, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, > {B38400 opost isig icanon echo ...}) = 0 > ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, > {B38400 opost isig icanon echo ...}) = 0 > getuid32() = 0 > geteuid32() = 0 > brk(0x9334b0) = 0x9334b0 > brk(0x934000) = 0x934000 > umask(0) = 022 > umask(022) = 0 > ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, > {B38400 opost isig icanon echo ...}) = 0 > ioctl(0, TIOCGWINSZ, {ws_row=0, ws_col=0, ws_xpixel=0, ws_ypixel=0}) = 0 > ioctl(1, TIOCGWINSZ, {ws_row=0, ws_col=0, ws_xpixel=0, ws_ypixel=0}) = 0 > ioctl(2, TIOCGWINSZ, {ws_row=0, ws_col=0, ws_xpixel=0, ws_ypixel=0}) = 0 > newfstatat(AT_FDCWD, ".", {st_mode=S_IFDIR|0755, st_size=1024, ...}, 0) = 0 > open(".", O_RDONLY|O_LARGEFILE) = 3 > dup(3) = 4 > fstat64(4, {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0 > fcntl64(4, F_GETFL) = 0x20000 (flags > O_RDONLY|O_LARGEFILE) > getdents64(4, /* 4 entries */, 1024) = 120 > newfstatat(4, ".", {st_mode=S_IFDIR|0755, st_size=1024, ...}, > AT_SYMLINK_NOFOLLOW) = 0 > newfstatat(4, "strace-armv5l", {st_mode=S_IFREG|0755, st_size=329448, ...}, > AT_SYMLINK_NOFOLLOW) = 0 > newfstatat(4, "..", {st_mode=S_IFDIR|0755, st_size=163, ...}, > AT_SYMLINK_NOFOLLOW) = 0 > newfstatat(4, "lost+found", {st_mode=S_IFDIR|0700, st_size=12288, ...}, > AT_SYMLINK_NOFOLLOW) = 0 > getdents64(4, /* 0 entries */, 1024) = 0 > close(4) = 0 > gettid() = 49 > tgkill(49, 49, SIGFPE) = 0 > --- SIGFPE {si_signo=SIGFPE, si_code=SI_TKILL, si_pid=49, si_uid=0} --- > +++ killed by SIGFPE +++ > Floating point exception > > So it's getting a ways into ls. Looks like it manages the whole dirtree and > then dies on the way back. Hmmm... > > Thanks for the heads up, I'm going to chew on these for a bit... Cool. And thanks for the very quick response! I tried building toybox for my development host. No problem with "ls" there. Now when I get toybox rebuilt with debug symbols I can figure out (from the core file) where the bug is. Could it be some wrapping of integer types on arm (that do not happen in x86) that cause division by zero?! I'm curious.... Best regards, Bj?rn Forsman 1356560586.0 From bjorn.forsman at gmail.com Wed Dec 26 23:02:53 2012 From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=) Date: Thu, 27 Dec 2012 00:02:53 +0100 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: References: <1356557795.31232.5@driftwood> Message-ID: On 26 December 2012 23:23, Bj?rn Forsman wrote: [...] > Now when I get toybox rebuilt with debug symbols I can figure out > (from the core file) where the bug is. I built toybox in a ./dev-environment.sh and caused a coredump with "./toybox_unstripped ls". Then I exited and mounted hda.sqf to mnt_rootfs/ and hdb.img to mnt_home/. This my backtrace: bfo at knopfler ~/aboriginal/build/system-image-armv5l $ sudo $(which arm-none-linux-gnueabi-gdb) mnt_home/toybox-0.4.2/toybox_unstripped mnt_home/toybox-0.4.2/core GNU gdb (Sourcery CodeBench Lite 2012.03-57) 7.2.50.20100908-cvs Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=i686-pc-linux-gnu --target=arm-none-linux-gnueabi". For bug reporting instructions, please see: ... Reading symbols from /home/bfo/aboriginal/build/system-image-armv5l/mnt_home/toybox-0.4.2/toybox_unstripped...done. warning: core file may not match specified executable file. [New LWP 1886] warning: Could not load shared library symbols for 6 libraries, e.g. /lib/libutil.so.0. Use the "info sharedlibrary" command to see the complete listing. Do you need "set solib-search-path" or "set sysroot"? warning: Unable to find dynamic linker. GDB will be unable to debug shared library initializers and track explicitly loaded dynamic code. Core was generated by `./toybox_unstripped ls'. Program terminated with signal 8, Arithmetic exception. #0 0xb6edbd34 in ?? () (gdb) set sysroot mnt_rootfs/ Reading symbols from mnt_rootfs/lib/libutil.so.0...(no debugging symbols found)...done. Loaded symbols for mnt_rootfs/lib/libutil.so.0 Reading symbols from mnt_rootfs/lib/libcrypt.so.0...(no debugging symbols found)...done. Loaded symbols for mnt_rootfs/lib/libcrypt.so.0 Reading symbols from mnt_rootfs/lib/libm.so.0...(no debugging symbols found)...done. Loaded symbols for mnt_rootfs/lib/libm.so.0 Reading symbols from mnt_rootfs/lib/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for mnt_rootfs/lib/libgcc_s.so.1 Reading symbols from mnt_rootfs/lib/libc.so.0...(no debugging symbols found)...done. Loaded symbols for mnt_rootfs/lib/libc.so.0 Reading symbols from mnt_rootfs/lib/ld-uClibc.so.0...(no debugging symbols found)...done. Loaded symbols for mnt_rootfs/lib/ld-uClibc.so.0 (gdb) bt #0 0xb6edbd34 in raise () from mnt_rootfs/lib/libc.so.0 #1 0x00028d84 in __div0 () at /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:1000 #2 0x000280c8 in __udivsi3 () at /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:754 #3 0x0001a1d0 in next_column (ul=0, dtlen=16, columns=0, xpos=0xbee639d0) at toys/posix/ls.c:186 #4 0x0001a8b8 in listfiles (dirfd=3, indir=0x6b5580) at toys/posix/ls.c:308 #5 0x0001a420 in listfiles (dirfd=-100, indir=0x6b54f8) at toys/posix/ls.c:228 #6 0x0001b3a0 in ls_main () at toys/posix/ls.c:455 #7 0x0000b224 in toy_exec (argv=0xbee63e48) at main.c:104 #8 0x0000b2b0 in toybox_main () at main.c:119 #9 0x0000b224 in toy_exec (argv=0xbee63e44) at main.c:104 #10 0x0000b2b0 in toybox_main () at main.c:119 #11 0x0000b4a4 in main (argc=2, argv=0xbee63e44) at main.c:161 (gdb) Let me know if you see something in there. I'll probably take a deeper look at this tomorrow. Best regards, Bj?rn Forsman 1356562973.0 From bjorn.forsman at gmail.com Wed Dec 26 23:44:19 2012 From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=) Date: Thu, 27 Dec 2012 00:44:19 +0100 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: References: <1356557795.31232.5@driftwood> Message-ID: On 27 December 2012 00:02, Bj?rn Forsman wrote: > On 26 December 2012 23:23, Bj?rn Forsman wrote: > [...] >> Now when I get toybox rebuilt with debug symbols I can figure out >> (from the core file) where the bug is. > > I built toybox in a ./dev-environment.sh and caused a coredump with > "./toybox_unstripped ls". Then I exited and mounted hda.sqf to > mnt_rootfs/ and hdb.img to mnt_home/. This my backtrace: > > bfo at knopfler ~/aboriginal/build/system-image-armv5l $ sudo $(which > arm-none-linux-gnueabi-gdb) mnt_home/toybox-0.4.2/toybox_unstripped > mnt_home/toybox-0.4.2/core > GNU gdb (Sourcery CodeBench Lite 2012.03-57) 7.2.50.20100908-cvs > Copyright (C) 2010 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "--host=i686-pc-linux-gnu > --target=arm-none-linux-gnueabi". > For bug reporting instructions, please see: > ... > Reading symbols from > /home/bfo/aboriginal/build/system-image-armv5l/mnt_home/toybox-0.4.2/toybox_unstripped...done. > > warning: core file may not match specified executable file. > [New LWP 1886] > > warning: Could not load shared library symbols for 6 libraries, e.g. > /lib/libutil.so.0. > Use the "info sharedlibrary" command to see the complete listing. > Do you need "set solib-search-path" or "set sysroot"? > > warning: Unable to find dynamic linker. > GDB will be unable to debug shared library initializers > and track explicitly loaded dynamic code. > Core was generated by `./toybox_unstripped ls'. > Program terminated with signal 8, Arithmetic exception. > #0 0xb6edbd34 in ?? () > (gdb) set sysroot mnt_rootfs/ > Reading symbols from mnt_rootfs/lib/libutil.so.0...(no debugging > symbols found)...done. > Loaded symbols for mnt_rootfs/lib/libutil.so.0 > Reading symbols from mnt_rootfs/lib/libcrypt.so.0...(no debugging > symbols found)...done. > Loaded symbols for mnt_rootfs/lib/libcrypt.so.0 > Reading symbols from mnt_rootfs/lib/libm.so.0...(no debugging symbols > found)...done. > Loaded symbols for mnt_rootfs/lib/libm.so.0 > Reading symbols from mnt_rootfs/lib/libgcc_s.so.1...(no debugging > symbols found)...done. > Loaded symbols for mnt_rootfs/lib/libgcc_s.so.1 > Reading symbols from mnt_rootfs/lib/libc.so.0...(no debugging symbols > found)...done. > Loaded symbols for mnt_rootfs/lib/libc.so.0 > Reading symbols from mnt_rootfs/lib/ld-uClibc.so.0...(no debugging > symbols found)...done. > Loaded symbols for mnt_rootfs/lib/ld-uClibc.so.0 > (gdb) bt > #0 0xb6edbd34 in raise () from mnt_rootfs/lib/libc.so.0 > #1 0x00028d84 in __div0 () at > /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:1000 > #2 0x000280c8 in __udivsi3 () at > /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:754 > #3 0x0001a1d0 in next_column (ul=0, dtlen=16, columns=0, > xpos=0xbee639d0) at toys/posix/ls.c:186 > #4 0x0001a8b8 in listfiles (dirfd=3, indir=0x6b5580) at toys/posix/ls.c:308 > #5 0x0001a420 in listfiles (dirfd=-100, indir=0x6b54f8) at toys/posix/ls.c:228 > #6 0x0001b3a0 in ls_main () at toys/posix/ls.c:455 > #7 0x0000b224 in toy_exec (argv=0xbee63e48) at main.c:104 > #8 0x0000b2b0 in toybox_main () at main.c:119 > #9 0x0000b224 in toy_exec (argv=0xbee63e44) at main.c:104 > #10 0x0000b2b0 in toybox_main () at main.c:119 > #11 0x0000b4a4 in main (argc=2, argv=0xbee63e44) at main.c:161 > (gdb) > > Let me know if you see something in there. I'll probably take a deeper > look at this tomorrow. ...I couldn't wait. (gdb) up #3 0x0001a1d0 in next_column (ul=0, dtlen=16, columns=0, xpos=0xbee639d0) at toys/posix/ls.c:186 186 height = (dtlen+columns-1)/columns; So this is where the SIGFPE/division-by-zero error comes from: columns=0. And columns=0 comes from listfiles(): (gdb) up #4 0x0001a8b8 in listfiles (dirfd=3, indir=0x6b5580) at toys/posix/ls.c:308 308 unsigned long next = next_column(ul, dtlen, columns, &curcol); And columns gets its zero value from this line, inside listfiles(): if (columns > TT.screen_width/2) columns = TT.screen_width/2; On my system (and yours too, apparently), TT.screen_width is zero! (Why?) I'm not sure what is the proper fix for this, as I'm not familiar with this code. Should next_column() simply ensure that columns>=1 or something? Or shouldn't that zero be passed on in the first place? In general, I don't like the idea of functions blowing up because they receive something "unexpected". Well, now I really must go to bed. More hacking tomorrow. Best regards, Bj?rn Forsman 1356565459.0 From rob at landley.net Wed Dec 26 23:50:01 2012 From: rob at landley.net (Rob Landley) Date: Wed, 26 Dec 2012 17:50:01 -0600 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: (from bjorn.forsman@gmail.com on Wed Dec 26 16:23:06 2012) References: <1356557795.31232.5@driftwood> Message-ID: <1356565801.31232.6@driftwood> On 12/26/2012 04:23:06 PM, Bj?rn Forsman wrote: > On 26 December 2012 22:36, Rob Landley wrote: > > Very wierd. Lemme see if I can reproduce that... yes I can. And > wow, you're > > right, it's only happening for dev-environment.sh which should be > TOTALLY > > unrelated... > > > > wget http://landley.net/aboriginal/strace-armv5l > > That returns 404 error code for me (but ignore that). Sorry, http://landley.net/aboriginal/bin/strace-armv5l (typed by hand, got it wrong). > > chmod +x strace-armv5l > > ./strace /bin/ls > > > > execve("/bin/ls", ["/bin/ls"], [/* 9 vars */]) = 0 > > brk(0) = 0x932000 > > brk(0x9324b0) = 0x9324b0 > > set_tls(0x932490, 0x5124c, 0, 0x1, 0x65fac) = 0 > > ioctl(0, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or > TCGETS, > > {B38400 opost isig icanon echo ...}) = 0 > > ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or > TCGETS, > > {B38400 opost isig icanon echo ...}) = 0 > > getuid32() = 0 > > geteuid32() = 0 > > brk(0x9334b0) = 0x9334b0 > > brk(0x934000) = 0x934000 > > umask(0) = 022 > > umask(022) = 0 > > ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or > TCGETS, > > {B38400 opost isig icanon echo ...}) = 0 > > ioctl(0, TIOCGWINSZ, {ws_row=0, ws_col=0, ws_xpixel=0, > ws_ypixel=0}) = 0 > > ioctl(1, TIOCGWINSZ, {ws_row=0, ws_col=0, ws_xpixel=0, > ws_ypixel=0}) = 0 > > ioctl(2, TIOCGWINSZ, {ws_row=0, ws_col=0, ws_xpixel=0, > ws_ypixel=0}) = 0 > > newfstatat(AT_FDCWD, ".", {st_mode=S_IFDIR|0755, st_size=1024, > ...}, 0) = 0 > > open(".", O_RDONLY|O_LARGEFILE) = 3 > > dup(3) = 4 > > fstat64(4, {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0 > > fcntl64(4, F_GETFL) = 0x20000 (flags > > O_RDONLY|O_LARGEFILE) > > getdents64(4, /* 4 entries */, 1024) = 120 > > newfstatat(4, ".", {st_mode=S_IFDIR|0755, st_size=1024, ...}, > > AT_SYMLINK_NOFOLLOW) = 0 > > newfstatat(4, "strace-armv5l", {st_mode=S_IFREG|0755, > st_size=329448, ...}, > > AT_SYMLINK_NOFOLLOW) = 0 > > newfstatat(4, "..", {st_mode=S_IFDIR|0755, st_size=163, ...}, > > AT_SYMLINK_NOFOLLOW) = 0 > > newfstatat(4, "lost+found", {st_mode=S_IFDIR|0700, st_size=12288, > ...}, > > AT_SYMLINK_NOFOLLOW) = 0 > > getdents64(4, /* 0 entries */, 1024) = 0 > > close(4) = 0 > > gettid() = 49 > > tgkill(49, 49, SIGFPE) = 0 > > --- SIGFPE {si_signo=SIGFPE, si_code=SI_TKILL, si_pid=49, si_uid=0} > --- > > +++ killed by SIGFPE +++ > > Floating point exception > > > > So it's getting a ways into ls. Looks like it manages the whole > dirtree and > > then dies on the way back. Hmmm... > > > > Thanks for the heads up, I'm going to chew on these for a bit... > > Cool. And thanks for the very quick response! Coincidence I happened to be reading at the time. My email sometimes has a day or two of latency before I catch up. Too many mailing lists, plus twitter, plus other things competing for time. > I tried building toybox for my development host. No problem with "ls" > there. Yeah, it works fine on i686. I'm poking at the other problem first though. > Now when I get toybox rebuilt with debug symbols I can figure out > (from the core file) where the bug is. Could it be some wrapping of > integer types on arm (that do not happen in x86) that cause division > by zero?! I'm curious.... Obviously I screwed something up. I'll post when I find it. Real life still intruding a bit. :) Rob 1356565801.0 From rob at landley.net Thu Dec 27 01:43:08 2012 From: rob at landley.net (Rob Landley) Date: Wed, 26 Dec 2012 19:43:08 -0600 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: (from bjorn.forsman@gmail.com on Wed Dec 26 17:02:53 2012) Message-ID: <1356572588.31232.7@driftwood> On 12/26/2012 05:02:53 PM, Bj?rn Forsman wrote: > (gdb) bt > #0 0xb6edbd34 in raise () from mnt_rootfs/lib/libc.so.0 > #1 0x00028d84 in __div0 () at > /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:1000 > #2 0x000280c8 in __udivsi3 () at > /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:754 > #3 0x0001a1d0 in next_column (ul=0, dtlen=16, columns=0, Ah. It's the -C support (which is now the default behavior, new change last release). Which implies terminal_size() is setting TT.screen_width to an insane value (either 0 or 1), since columns is capped at TT.screen_width/2 and then we do a modulus on columns to find our rows and there's the division by zero error. Right. And since 0 is filtered out in terminal_size() in lib/lib.c, this implies that ioctl(TIOCGWINSZ) is setting the screen width to 1. (Or something is setting COLUMNS=0 which would override the check.) Either way: http://landley.net/hg/toybox/rev/017b8fd3c9ac Not sure that's the proper fix but it'll do for now. (I may need to tweak the library function to never return insane values, but I need to add ascii probe logic ot it anyway.) Thanks. I'm back poking at the other segfault now... Rob 1356572588.0 From rob at landley.net Thu Dec 27 06:37:46 2012 From: rob at landley.net (Rob Landley) Date: Thu, 27 Dec 2012 00:37:46 -0600 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: (from bjorn.forsman@gmail.com on Wed Dec 26 17:44:19 2012) Message-ID: <1356590266.31232.8@driftwood> On 12/26/2012 05:44:19 PM, Bj?rn Forsman wrote: > On 27 December 2012 00:02, Bj?rn Forsman > wrote: > > On 26 December 2012 23:23, Bj?rn Forsman > wrote: > > [...] > >> Now when I get toybox rebuilt with debug symbols I can figure out > >> (from the core file) where the bug is. > > > > I built toybox in a ./dev-environment.sh and caused a coredump with > > "./toybox_unstripped ls". Then I exited and mounted hda.sqf to > > mnt_rootfs/ and hdb.img to mnt_home/. This my backtrace: > > > > bfo at knopfler ~/aboriginal/build/system-image-armv5l $ sudo $(which > > arm-none-linux-gnueabi-gdb) mnt_home/toybox-0.4.2/toybox_unstripped > > mnt_home/toybox-0.4.2/core > > GNU gdb (Sourcery CodeBench Lite 2012.03-57) 7.2.50.20100908-cvs > > Copyright (C) 2010 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later > > > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > > and "show warranty" for details. > > This GDB was configured as "--host=i686-pc-linux-gnu > > --target=arm-none-linux-gnueabi". > > For bug reporting instructions, please see: > > ... > > Reading symbols from > > > /home/bfo/aboriginal/build/system-image-armv5l/mnt_home/toybox-0.4.2/toybox_unstripped...done. > > > > warning: core file may not match specified executable file. > > [New LWP 1886] > > > > warning: Could not load shared library symbols for 6 libraries, e.g. > > /lib/libutil.so.0. > > Use the "info sharedlibrary" command to see the complete listing. > > Do you need "set solib-search-path" or "set sysroot"? > > > > warning: Unable to find dynamic linker. > > GDB will be unable to debug shared library initializers > > and track explicitly loaded dynamic code. > > Core was generated by `./toybox_unstripped ls'. > > Program terminated with signal 8, Arithmetic exception. > > #0 0xb6edbd34 in ?? () > > (gdb) set sysroot mnt_rootfs/ > > Reading symbols from mnt_rootfs/lib/libutil.so.0...(no debugging > > symbols found)...done. > > Loaded symbols for mnt_rootfs/lib/libutil.so.0 > > Reading symbols from mnt_rootfs/lib/libcrypt.so.0...(no debugging > > symbols found)...done. > > Loaded symbols for mnt_rootfs/lib/libcrypt.so.0 > > Reading symbols from mnt_rootfs/lib/libm.so.0...(no debugging > symbols > > found)...done. > > Loaded symbols for mnt_rootfs/lib/libm.so.0 > > Reading symbols from mnt_rootfs/lib/libgcc_s.so.1...(no debugging > > symbols found)...done. > > Loaded symbols for mnt_rootfs/lib/libgcc_s.so.1 > > Reading symbols from mnt_rootfs/lib/libc.so.0...(no debugging > symbols > > found)...done. > > Loaded symbols for mnt_rootfs/lib/libc.so.0 > > Reading symbols from mnt_rootfs/lib/ld-uClibc.so.0...(no debugging > > symbols found)...done. > > Loaded symbols for mnt_rootfs/lib/ld-uClibc.so.0 > > (gdb) bt > > #0 0xb6edbd34 in raise () from mnt_rootfs/lib/libc.so.0 > > #1 0x00028d84 in __div0 () at > > > /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:1000 > > #2 0x000280c8 in __udivsi3 () at > > > /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:754 > > #3 0x0001a1d0 in next_column (ul=0, dtlen=16, columns=0, > > xpos=0xbee639d0) at toys/posix/ls.c:186 > > #4 0x0001a8b8 in listfiles (dirfd=3, indir=0x6b5580) at > toys/posix/ls.c:308 > > #5 0x0001a420 in listfiles (dirfd=-100, indir=0x6b54f8) at > toys/posix/ls.c:228 > > #6 0x0001b3a0 in ls_main () at toys/posix/ls.c:455 > > #7 0x0000b224 in toy_exec (argv=0xbee63e48) at main.c:104 > > #8 0x0000b2b0 in toybox_main () at main.c:119 > > #9 0x0000b224 in toy_exec (argv=0xbee63e44) at main.c:104 > > #10 0x0000b2b0 in toybox_main () at main.c:119 > > #11 0x0000b4a4 in main (argc=2, argv=0xbee63e44) at main.c:161 > > (gdb) > > > > Let me know if you see something in there. I'll probably take a > deeper > > look at this tomorrow. > > ...I couldn't wait. > > (gdb) up > #3 0x0001a1d0 in next_column (ul=0, dtlen=16, columns=0, > xpos=0xbee639d0) at toys/posix/ls.c:186 > 186 height = (dtlen+columns-1)/columns; > > So this is where the SIGFPE/division-by-zero error comes from: > columns=0. > > And columns=0 comes from listfiles(): > > (gdb) up > #4 0x0001a8b8 in listfiles (dirfd=3, indir=0x6b5580) at > toys/posix/ls.c:308 > 308 unsigned long next = next_column(ul, dtlen, columns, > &curcol); > > And columns gets its zero value from this line, inside listfiles(): > > if (columns > TT.screen_width/2) columns = TT.screen_width/2; > > On my system (and yours too, apparently), TT.screen_width is zero! > (Why?) Because /dev/console hooks up to a serial device, and serial devices don't inherently have a width or height. (There's no attached display device, even though it's a console. Consoles do multiple things and this hardware only supports some of them, so the rest are stubbed out. Unfortunately, they're not stubbed out _consistently_ across different drivers...) I see another part of the problem, which is that this code: http://landley.net/hg/toybox/file/017b8fd3c9ac/lib/lib.c#l936 The parts filtering out x=0 or y=0 aren't vetoing zero values from the ioctl, they're checking the _pointer_ for zero and not recording it. (I wrote that code in 2007, the details are fuzzy.) > I'm not sure what is the proper fix for this, as I'm not familiar with > this code. Should next_column() simply ensure that columns>=1 or > something? Or shouldn't that zero be passed on in the first place? In > general, I don't like the idea of functions blowing up because they > receive something "unexpected". The zero should be detected and we'd have additional fallbacks, such as: http://lists.busybox.net/pipermail/busybox/2009-May/069296.html Which is non-obvious because of: http://lists.busybox.net/pipermail/busybox/2008-October/067348.html http://lists.busybox.net/pipermail/busybox/2008-October/067418.html Which is why it's still a todo item. In theory if I get toysh doing this right, then it can pass the data along to vi and less and such, although if you resize a terminal it would have to re-probe (and since it doesn't _know_ it would have to do so, it would have to do so fairly frequently). The problem with those is other data the user legitimately typed is coming in, potentially before and after the escape probe, so you've gotta filter all incoming data and pick out the probe response. If you're already parsing cursor keys this is easy. Since ls isn't doiing any of that, probing would just be awkward so toysh should set COLUMNS to a sane value. The problem is, we're not using toysh yet because it's still just a stub... Rob 1356590266.0 From rajeev_v_pillai at yahoo.com Thu Dec 27 00:53:51 2012 From: rajeev_v_pillai at yahoo.com (Rajeev V. Pillai) Date: Thu, 27 Dec 2012 08:53:51 +0800 (SGT) Subject: [Aboriginal] uClibc-0.9.33.2 statfs() does not populate the `f_frsize' field of `struct statfs' Message-ID: <1356569631.69654.YahooMailNeo@web190706.mail.sg3.yahoo.com> This bug has already been reported to the uClibc Bugzilla system. On uClibc-0.9.33.2, statfs() does not populate the `f_frsize' field of `struct statfs' even though it defines _STATFS_F_FRSIZE. /* BEGIN test code */ #define _FILE_OFFSET_BITS 64 #include #include #include #include #include int main(int argc, char* argv[]) { struct statfs s; int i; for (i = 1; i < argc; i++) { if (statfs(argv[i], &s) != 0) { fprintf(stderr, "%s: %s: statfs failed. %s\n", *argv, argv[i], strerror(errno)); exit(EXIT_FAILURE); } printf("%s:\n\tblocks=%lld\n\tblkfree=%lld\n\tbsize=%d\n", argv[i], s.f_blocks, s.f_bfree, s.f_bsize); #ifdef _STATFS_F_FRSIZE printf("\tfrsize=%d\n", s.f_frsize); /* RANDOM values printed */ #endif } exit(EXIT_SUCCESS); } /* END test code */ 1356569631.0 From onefang at gmail.com Thu Dec 27 06:56:07 2012 From: onefang at gmail.com (David Seikel) Date: Thu, 27 Dec 2012 16:56:07 +1000 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: <1356590266.31232.8@driftwood> References: <1356590266.31232.8@driftwood> Message-ID: <20121227165607.1cdfb66a.onefang@gmail.com> On Thu, 27 Dec 2012 00:37:46 -0600 Rob Landley wrote: > On 12/26/2012 05:44:19 PM, Bj?rn Forsman wrote: > > On 27 December 2012 00:02, Bj?rn Forsman > > wrote: > > > On 26 December 2012 23:23, Bj?rn Forsman > > > > > wrote: > > > [...] > > >> Now when I get toybox rebuilt with debug symbols I can figure out > > >> (from the core file) where the bug is. > > > > > > I built toybox in a ./dev-environment.sh and caused a coredump > > > with "./toybox_unstripped ls". Then I exited and mounted hda.sqf > > > to mnt_rootfs/ and hdb.img to mnt_home/. This my backtrace: > > > > > > bfo at knopfler ~/aboriginal/build/system-image-armv5l $ sudo $(which > > > arm-none-linux-gnueabi-gdb) > > > mnt_home/toybox-0.4.2/toybox_unstripped mnt_home/toybox-0.4.2/core > > > GNU gdb (Sourcery CodeBench Lite 2012.03-57) 7.2.50.20100908-cvs > > > Copyright (C) 2010 Free Software Foundation, Inc. > > > License GPLv3+: GNU GPL version 3 or later > > > > > This is free software: you are free to change and redistribute it. > > > There is NO WARRANTY, to the extent permitted by law. Type > > > "show > > copying" > > > and "show warranty" for details. > > > This GDB was configured as "--host=i686-pc-linux-gnu > > > --target=arm-none-linux-gnueabi". > > > For bug reporting instructions, please see: > > > ... > > > Reading symbols from > > > > > /home/bfo/aboriginal/build/system-image-armv5l/mnt_home/toybox-0.4.2/toybox_unstripped...done. > > > > > > warning: core file may not match specified executable file. > > > [New LWP 1886] > > > > > > warning: Could not load shared library symbols for 6 libraries, > > > e.g. /lib/libutil.so.0. > > > Use the "info sharedlibrary" command to see the complete listing. > > > Do you need "set solib-search-path" or "set sysroot"? > > > > > > warning: Unable to find dynamic linker. > > > GDB will be unable to debug shared library initializers > > > and track explicitly loaded dynamic code. > > > Core was generated by `./toybox_unstripped ls'. > > > Program terminated with signal 8, Arithmetic exception. > > > #0 0xb6edbd34 in ?? () > > > (gdb) set sysroot mnt_rootfs/ > > > Reading symbols from mnt_rootfs/lib/libutil.so.0...(no debugging > > > symbols found)...done. > > > Loaded symbols for mnt_rootfs/lib/libutil.so.0 > > > Reading symbols from mnt_rootfs/lib/libcrypt.so.0...(no debugging > > > symbols found)...done. > > > Loaded symbols for mnt_rootfs/lib/libcrypt.so.0 > > > Reading symbols from mnt_rootfs/lib/libm.so.0...(no debugging > > symbols > > > found)...done. > > > Loaded symbols for mnt_rootfs/lib/libm.so.0 > > > Reading symbols from mnt_rootfs/lib/libgcc_s.so.1...(no debugging > > > symbols found)...done. > > > Loaded symbols for mnt_rootfs/lib/libgcc_s.so.1 > > > Reading symbols from mnt_rootfs/lib/libc.so.0...(no debugging > > symbols > > > found)...done. > > > Loaded symbols for mnt_rootfs/lib/libc.so.0 > > > Reading symbols from mnt_rootfs/lib/ld-uClibc.so.0...(no debugging > > > symbols found)...done. > > > Loaded symbols for mnt_rootfs/lib/ld-uClibc.so.0 > > > (gdb) bt > > > #0 0xb6edbd34 in raise () from mnt_rootfs/lib/libc.so.0 > > > #1 0x00028d84 in __div0 () at > > > > > /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:1000 > > > #2 0x000280c8 in __udivsi3 () at > > > > > /home/bfo/aboriginal/build/temp-armv5l/gcc-core/gcc/config/arm/lib1funcs.asm:754 > > > #3 0x0001a1d0 in next_column (ul=0, dtlen=16, columns=0, > > > xpos=0xbee639d0) at toys/posix/ls.c:186 > > > #4 0x0001a8b8 in listfiles (dirfd=3, indir=0x6b5580) at > > toys/posix/ls.c:308 > > > #5 0x0001a420 in listfiles (dirfd=-100, indir=0x6b54f8) at > > toys/posix/ls.c:228 > > > #6 0x0001b3a0 in ls_main () at toys/posix/ls.c:455 > > > #7 0x0000b224 in toy_exec (argv=0xbee63e48) at main.c:104 > > > #8 0x0000b2b0 in toybox_main () at main.c:119 > > > #9 0x0000b224 in toy_exec (argv=0xbee63e44) at main.c:104 > > > #10 0x0000b2b0 in toybox_main () at main.c:119 > > > #11 0x0000b4a4 in main (argc=2, argv=0xbee63e44) at main.c:161 > > > (gdb) > > > > > > Let me know if you see something in there. I'll probably take a > > deeper > > > look at this tomorrow. > > > > ...I couldn't wait. > > > > (gdb) up > > #3 0x0001a1d0 in next_column (ul=0, dtlen=16, columns=0, > > xpos=0xbee639d0) at toys/posix/ls.c:186 > > 186 height = (dtlen+columns-1)/columns; > > > > So this is where the SIGFPE/division-by-zero error comes from: > > columns=0. > > > > And columns=0 comes from listfiles(): > > > > (gdb) up > > #4 0x0001a8b8 in listfiles (dirfd=3, indir=0x6b5580) at > > toys/posix/ls.c:308 > > 308 unsigned long next = next_column(ul, dtlen, > > columns, &curcol); > > > > And columns gets its zero value from this line, inside listfiles(): > > > > if (columns > TT.screen_width/2) columns = TT.screen_width/2; > > > > On my system (and yours too, apparently), TT.screen_width is zero! > > (Why?) > > Because /dev/console hooks up to a serial device, and serial devices > don't inherently have a width or height. (There's no attached > display device, even though it's a console. Consoles do multiple > things and this hardware only supports some of them, so the rest are > stubbed out. Unfortunately, they're not stubbed out _consistently_ > across different drivers...) > > I see another part of the problem, which is that this code: > > http://landley.net/hg/toybox/file/017b8fd3c9ac/lib/lib.c#l936 > > The parts filtering out x=0 or y=0 aren't vetoing zero values from > the ioctl, they're checking the _pointer_ for zero and not recording > it. (I wrote that code in 2007, the details are fuzzy.) > > > I'm not sure what is the proper fix for this, as I'm not familiar > > with this code. Should next_column() simply ensure that columns>=1 > > or something? Or shouldn't that zero be passed on in the first > > place? In general, I don't like the idea of functions blowing up > > because they receive something "unexpected". > > The zero should be detected and we'd have additional fallbacks, such > as: > > http://lists.busybox.net/pipermail/busybox/2009-May/069296.html > > Which is non-obvious because of: > http://lists.busybox.net/pipermail/busybox/2008-October/067348.html > http://lists.busybox.net/pipermail/busybox/2008-October/067418.html > > Which is why it's still a todo item. In theory if I get toysh doing > this right, then it can pass the data along to vi and less and such, > although if you resize a terminal it would have to re-probe (and > since it doesn't _know_ it would have to do so, it would have to do > so fairly frequently). The problem with those is other data the user > legitimately typed is coming in, potentially before and after the > escape probe, so you've gotta filter all incoming data and pick out > the probe response. If you're already parsing cursor keys this is > easy. > > Since ls isn't doiing any of that, probing would just be awkward so > toysh should set COLUMNS to a sane value. The problem is, we're not > using toysh yet because it's still just a stub... I got some or all of that terminal sizing stuff in my editor thingy already if I remember correctly. I remember you wanted me to cut it down into tiny bits to start with, so you don't have to digest the huge lump I sent. Two problems - A) I wanted to get most of it working to sort out HOW I would get it all working, before cutting it down to small bite sized bits later. B) had no time to do anything with it since I posted the big blob, a new contract was keeping me busy. So the next step was to cut it down into something you could get your mind around easily, but that B) thingy stopped me from getting that done. While the new contract is taking a break for the Christmas / New Year holiday season (that's common here), the old contract decided this is the perfect time to keep me busy (that bastard). sigh -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From rob at landley.net Thu Dec 27 09:28:43 2012 From: rob at landley.net (Rob Landley) Date: Thu, 27 Dec 2012 03:28:43 -0600 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: (from bjorn.forsman@gmail.com on Wed Dec 26 16:23:06 2012) Message-ID: <1356600523.31232.9@driftwood> On 12/26/2012 04:23:06 PM, Bj?rn Forsman wrote: > On 26 December 2012 22:36, Rob Landley wrote: > > On 12/26/2012 02:55:04 PM, Bj?rn Forsman wrote: > >> Not sure what exactly is segfaulting, but segfault == bug to me. > And > >> it only happens with ./run-emulator.sh, not ./dev-environment.sh. > > > > There are somewhat different code paths in the init.sh shell script > so it's > > probably something only running in the first case. I need to > rebuild the > > armv5l target to test this, that'll take a few minutes... Huh. I did a set -x at the start of init.sh and the relevant chunk is: + mountpoint -q dev/pts + mount -t devpts dev/pts dev/pts + '[' -z '' ']' ++ wc -w ++ echo /sys/devices/system/cpu/cpu0 Segmentation fault + export CPUS=1 + CPUS=1 Which corresponds to: # If nobody said how many CPUS to use in builds, try to figure it out. if [ -z "$CPUS" ] then export CPUS=$(echo /sys/devices/system/cpu/cpu[0-9]* | wc -w) [ "$CPUS" -lt 1 ] && CPUS=1 fi export PS1='($HOST:$CPUS) \w \$ ' But... the export command is getting CPUS=1. The next line isn't an export (because it assumes the previous line exported the variable even if it got the wrong value, which it can't because the echo would still output a string even if the wildcard didn't find any files so that's an unnecessary test). So wc -w had to output "1", meaning the echo fed it a string, meaning both _worked_. Either they segfaulted on exit (why?) or something's funky in the shell... I tried this on the command line: unset CPUS; if [ -z "$CPUS" ]; then export CPUS=$(echo /sys/devices/system/cpu/cpu[0-9]* | wc -w); [ "$CPUS" -lt 1 ] && CPUS=1; fi And it didn't segfault. Ok, replace the export CPUS line in the script with: export CPUS=$(printf 0 | wc -w) Still segfaults. export CPUS=$(printf 0 | cat) Still segfaults... And that's starting to imply that it's the _shell_. Is this a bug in ash? Ok, revert the changes and instead make the first line #!/bin/bash and... Still segfaults? Weeeeeeird. Ok, export CPUS=1 with the rest of the $() commented out... no segfault. Try export CPUS=$(echo 1) and it segfaults. What is this, somethign in the toybox setup/exit code? It'd have to be the exit code because the segfault is _after_ the command otherwise completes. Maybe that new setjmp/longjmp stuff? No, that's not the common exit path, only the error exit path, toy_exec() ends with: toys.which->toy_main(); if (fflush(NULL) || ferror(stdout)) perror_exit("write"); exit(toys.exitval); Ok, what's a command that isn't toybox... ah, expr is still busybox. Try $(expr 1) and it does _not_ segfault. Ok, now we're getting somewhere. Unless the fact busybox isn't forking an external program is part of the reproduction sequence? Ummm... echo /sys/devices/system/cpu/cpu[0-9]* | wc -w export CPUS=1 #$(echo /sys/devices/system/cpu/cpu[0-9]* | wc -w) echo zero export CPUS=$(expr 1) echo one export CPUS=$(hello-dynamic) echo two export CPUS=$(echo 1) echo three Gives us: Freeing init memory: 96K 1 zero one two Segmentation fault three 8139cp 0000:00:0c.0 eth0: link up, 100Mbps, full-duplex, lpa 0x05E1 Right, that's toybox. Will it do it on a second copy of toybox in the same filesystem? Yes. Ok, I can twiddle that copy without screwing up the base system relying on the rest. Now... printf("exitval=%d\n", toys.exitval); exit(toys.exitval); Goes: exitval=0 Segmentation fault Thats kind of unsporting. It looks like something I'm doing is screwing up the uClibc exit code. Let's try _exit(toys.exitval); and see if that avoids the segfault? Apparently not. So it's not the atexit() call chain or the stdio shutdown, it's something that toybox does that irritates ash even when we call the exit system call more or less directly. Sigh. I haven't taken a serious look at the ash source in _years_. And of course the first file I look at is shell/ash_ptr_hack.c and I remember WHY I haven't looked at this since I stopped actually having a duty to. Quick search for an alternative to sticking printf() calls into ash, how about running toybox under strace and see if that catches the segfault? Hang on, strace didn't run? Oh, duh, updated the wrong copy of the file. (Being a bit too clever not re-running build stages to speed up test cycles on my sad little netbook: there are 5 copies of init.sh in play right now... And strace says: write(1, "exitval=0\n", 10exitval=0 ) = 10 exit_group(0) = ? +++ exited with 0 +++ Segmentation fault two Which means it's happening after _strace_ exits. Wheee... Printf in ash time, but first I hit the grocery store. Rob 1356600523.0 From rob at landley.net Thu Dec 27 12:06:53 2012 From: rob at landley.net (Rob Landley) Date: Thu, 27 Dec 2012 06:06:53 -0600 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: <20121227165607.1cdfb66a.onefang@gmail.com> (from onefang@gmail.com on Thu Dec 27 00:56:07 2012) Message-ID: <1356610013.31232.10@driftwood> On 12/27/2012 12:56:07 AM, David Seikel wrote: > On Thu, 27 Dec 2012 00:37:46 -0600 Rob Landley > wrote: > > Since ls isn't doiing any of that, probing would just be awkward so > > toysh should set COLUMNS to a sane value. The problem is, we're not > > using toysh yet because it's still just a stub... > > I got some or all of that terminal sizing stuff in my editor thingy > already if I remember correctly. I remember you wanted me to cut it > down into tiny bits to start with, so you don't have to digest the > huge > lump I sent. Basically what I'd like is a self-contained line reader. More or less a getline variant that can handle cursoring left and right to insert stuff, handle backspace past a wordwrap (which on unix requires knowing the screen width and your current cursor position), and one that lets up/down escape sequences be hooked for less/more screen scrolling, vi line movement, shell command history, and so on. There's sort of three levels here, the first is "parse raw input, assembling escape sequences into cursor-left and similar as necessary". (That's the one I had to redo in busybox vi because they didn't do it right.) The second is "edit a line of text, handling cursoring _within_ the line". That's the better/interactive getline(). The third is handling escapes from that line triggering behavior in the larger command (cursor up and cursor down do different things in less, in vi, and in shell command history). The fiddly bit is that terminal_size() sort of has to integrate with all of these. Possibly I need to have width and height be members of struct toy_context. Or have the better_getline() take a pointer to a state structure it can update, containing that info... > Two problems - A) I wanted to get most of it working to > sort out HOW I would get it all working, before cutting it down to > small bite sized bits later. B) had no time to do anything with it > since I posted the big blob, a new contract was keeping me busy. I know the feeling. :) > So the next step was to cut it down into something you could get your > mind > around easily, but that B) thingy stopped me from getting that done. > > While the new contract is taking a break for the Christmas / New Year > holiday season (that's common here), the old contract decided this is > the perfect time to keep me busy (that bastard). sigh Eh, I've got my hands full right now anyway, juggling too many things as usual. (I wrote a python program to send non-whitespace-damaged email to linux-kernel so I could submit the perl removal patches upstream, and I didn't do it quite right so I've got it mostly updated to create proper message-ids and send later messages as replies to the first one, and I've been studying how to get a patch series into linux-next but I have to set up a git tree and dreamhost is being stroppy but they just migrated landley.net to a new server yesterday that might have a better time with it...) And I start a new contract of my own on the 21st, so I've only got another 3 weeks or so to bang away on open source stuff before my time gets chomped again for 6 months or so. (Shouldn't be as bad as last time, though.) Mostly I've been relaxing during the holidays, though. Fade got a dog from the shelter with _massive_ separation anxiety issues, my foot's still not recovered from https://twitter.com/landley/status/280546637565280256 (don't click on that unless you want to see a picture of me bleeding), and so on. Non-computer stuff takes time too, go figure. :) Rob 1356610013.0 From onefang at gmail.com Thu Dec 27 12:21:21 2012 From: onefang at gmail.com (David Seikel) Date: Thu, 27 Dec 2012 22:21:21 +1000 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: <1356610013.31232.10@driftwood> References: <20121227165607.1cdfb66a.onefang@gmail.com> <1356610013.31232.10@driftwood> Message-ID: <20121227222121.6ae0f528.onefang@gmail.com> On Thu, 27 Dec 2012 06:06:53 -0600 Rob Landley wrote: > On 12/27/2012 12:56:07 AM, David Seikel wrote: > > On Thu, 27 Dec 2012 00:37:46 -0600 Rob Landley > > wrote: > > > Since ls isn't doiing any of that, probing would just be awkward > > > so toysh should set COLUMNS to a sane value. The problem is, > > > we're not using toysh yet because it's still just a stub... > > > > I got some or all of that terminal sizing stuff in my editor thingy > > already if I remember correctly. I remember you wanted me to cut it > > down into tiny bits to start with, so you don't have to digest the > > huge > > lump I sent. > > Basically what I'd like is a self-contained line reader. More or less > a getline variant that can handle cursoring left and right to insert > stuff, handle backspace past a wordwrap (which on unix requires > knowing the screen width and your current cursor position), and one > that lets up/down escape sequences be hooked for less/more screen > scrolling, vi line movement, shell command history, and so on. Which is most of an editor already, so how to break it up into bite sized morsels? I'll think about it later, got paying work to do, then sleep. > There's sort of three levels here, the first is "parse raw input, > assembling escape sequences into cursor-left and similar as > necessary". (That's the one I had to redo in busybox vi because they > didn't do it right.) > > The second is "edit a line of text, handling cursoring _within_ the > line". That's the better/interactive getline(). > > The third is handling escapes from that line triggering behavior in > the larger command (cursor up and cursor down do different things in > less, in vi, and in shell command history). > > The fiddly bit is that terminal_size() sort of has to integrate with > all of these. Possibly I need to have width and height be members of > struct toy_context. Or have the better_getline() take a pointer to a > state structure it can update, containing that info... > > > Two problems - A) I wanted to get most of it working to > > sort out HOW I would get it all working, before cutting it down to > > small bite sized bits later. B) had no time to do anything with it > > since I posted the big blob, a new contract was keeping me busy. > > I know the feeling. :) -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From rob at landley.net Thu Dec 27 13:30:39 2012 From: rob at landley.net (Rob Landley) Date: Thu, 27 Dec 2012 07:30:39 -0600 Subject: [Aboriginal] Merry christmas, I have found two bugs In-Reply-To: (from bjorn.forsman@gmail.com on Wed Dec 26 16:23:06 2012) Message-ID: <1356615039.31232.11@driftwood> On 12/26/2012 04:23:06 PM, Bj?rn Forsman wrote: > On 26 December 2012 22:36, Rob Landley wrote: > > On 12/26/2012 02:55:04 PM, Bj?rn Forsman wrote: > [...] > >> And now I think I've found two bugs. Bug 1 is > >> a segmentation fault during system image boot: > >> > >> $ cd build/system-image-avmv5l > >> $ ./run-emulator.sh > >> [...] > >> Freeing init memory: 96K > >> Segmentation fault > >> 8139cp 0000:00:0c.0 eth0: link up, 100Mbps, full-duplex, lpa 0x05E1 > >> Not using distcc. > >> Type exit when done. > >> (armv5l:1) /home # > >> > >> Not sure what exactly is segfaulting, but segfault == bug to me. > And > >> it only happens with ./run-emulator.sh, not ./dev-environment.sh. > > > > There are somewhat different code paths in the init.sh shell script > so it's > > probably something only running in the first case. I need to > rebuild the > > armv5l target to test this, that'll take a few minutes... Still grinding away on this. It's... weird. If I run_emulator.sh with KERNEL_EXTRA=init=/bin/bash and then exec /sbin/init.sh it does _not_ segfault. It doesn't do so with init=/bin/bash or init=/bin/ash. But it _does_ do so when init=/sbin/init.sh when that starts with #!/bin/bash or #!/bin/ash. So it only happens when the script is run as the first process. Not just as PID 1, as the _first_process_. If you exec a shell that execs the script, it doesn't segfault. So the same thing happening with two different shells, but only from a very specific execution environment. Is there some asynchronous kernel thread that happens to damage whatever's running at the time it completes? Add a sleep 1" before the if CPU block... no difference. Back up, confirm whether ash or bash IS running, add an ls -l /proc/self/exe and it says... toybox. Sigh. Make that ls -l /proc/1/exe (ask about the _shell_ not the ls process) and it says... /bin/bash. And when I switch the #! line to ash it... isn't segfaulting? Hang on, according to "hg diff" the initial version wasn't #!/bin/ash it was #!/bin/sh which points to _bash_. So I switched it from bash to bash, which is an invalid test. So it's NOT a bug affecting both shells (and thus probably some libc thing with setsid or who knows what), it's a BASH BUG. Given how old the version of bash I'm using is, and that I've already had to fix deep obscure bugs in it before (the backstory behind http://landley.net/hg/aboriginal/rev/de8e06350227 is long and epic and described in my blog over a course of weeks), I think the correct fix is #!/bin/ash (and eventually #!/bin/toysh). Rob 1356615039.0 From rob at landley.net Thu Dec 27 14:54:53 2012 From: rob at landley.net (Rob Landley) Date: Thu, 27 Dec 2012 08:54:53 -0600 Subject: [Aboriginal] uClibc-0.9.33.2 statfs() does not populate the `f_frsize' field of `struct statfs' In-Reply-To: <1356569631.69654.YahooMailNeo@web190706.mail.sg3.yahoo.com> (from rajeev_v_pillai@yahoo.com on Wed Dec 26 18:53:51 2012) Message-ID: <1356620093.31232.13@driftwood> On 12/26/2012 06:53:51 PM, Rajeev V. Pillai wrote: > This bug has already been reported to the uClibc Bugzilla system. > > On uClibc-0.9.33.2, statfs() does not populate the `f_frsize' field > of `struct > statfs' even though it defines _STATFS_F_FRSIZE. On which architecture? The code in libc/sysdeps/linux/common/statfs.c is just a trivial syscall wrapper, there's no marshalling of data back and forth between kernel/user structures that I noticed. But libc/misc/statfs/statfs64.c is crazy and stupid and has no reason to exist, what on earth is it doing? Why is it doing it? The mind boggles. Right, I can copy one more field in this pointless data marshalling wrapper. (Long-term: migrating to musl.) In the meantime, something like the attached? (What's your use case, by the way? What package did this break?) Rob -------------- next part -------------- A non-text attachment was scrubbed... Name: uClibc-statfs.patch Type: text/x-patch Size: 504 bytes Desc: not available URL: From rajeev_v_pillai at yahoo.com Thu Dec 27 21:03:32 2012 From: rajeev_v_pillai at yahoo.com (Rajeev V. Pillai) Date: Fri, 28 Dec 2012 05:03:32 +0800 (SGT) Subject: [Aboriginal] uClibc-0.9.33.2 statfs() does not populate the `f_frsize' field of `struct statfs' In-Reply-To: <1356620093.31232.13@driftwood> References: <1356569631.69654.YahooMailNeo@web190706.mail.sg3.yahoo.com> (from rajeev_v_pillai@yahoo.com on Wed Dec 26 18:53:51 2012) <1356620093.31232.13@driftwood> Message-ID: <1356642212.92639.YahooMailNeo@web190703.mail.sg3.yahoo.com> > From: Rob Landley > > On which architecture? > ARM Tegra 2 tablet (armv7-a) running Android 2.3.5. > Right, I can copy one more field in this pointless data marshalling wrapper. > (Long-term: migrating to musl.) In the meantime, something like the attached? > Yeah. Either that or zero it out as the statfs() manpage says should be done. BTW, musl has a similar data marshalling wrapper. > (What's your use case, by the way? What package did this break?) > None, except my version of df(1) which I wrote to test my own *mntent() functions (missing in Android's Bionic libc). I wanted to use the POSIX statvfs(3) whose man page states that the various block-related statistics were in units of `f_frsize' rather than `f_bsize'. But, since Android's Bionic doesn't have statvfs(3), but, has statfs(2), I decided to go with that latter. However, my manpage of statfs(2) doesn't make clear whether the units reported are by statfs() are in `f_frsize' or `f_bsize'. I decided to go with `f_frsize' given that `f_bsize' is officially the ``optimal transfer block size''. I noticed that both busybox and toybox use `f_bsize', however--which doesn't make much difference on most (ie. all those not implementing block fragments) filesystems, though. 1356642212.0 From rob at landley.net Fri Dec 28 04:07:34 2012 From: rob at landley.net (Rob Landley) Date: Thu, 27 Dec 2012 22:07:34 -0600 Subject: [Aboriginal] uClibc-0.9.33.2 statfs() does not populate the `f_frsize' field of `struct statfs' In-Reply-To: <1356642212.92639.YahooMailNeo@web190703.mail.sg3.yahoo.com> (from rajeev_v_pillai@yahoo.com on Thu Dec 27 15:03:32 2012) Message-ID: <1356667654.31232.16@driftwood> On 12/27/2012 03:03:32 PM, Rajeev V. Pillai wrote: > > From: Rob Landley > > > > On which architecture? > > > > ARM Tegra 2 tablet (armv7-a) running Android 2.3.5. > > > > Right, I can copy one more field in this pointless data marshalling > wrapper. > > (Long-term: migrating to musl.) In the meantime, something like the > attached? > > > Yeah. Either that or zero it out as the statfs() manpage says should > be done. > BTW, musl has a similar data marshalling wrapper. > > > > (What's your use case, by the way? What package did this break?) > > > None, except my version of df(1) which I wrote to test my own > *mntent() functions > (missing in Android's Bionic libc). > > I wanted to use the POSIX statvfs(3) whose man page states that the > various > block-related statistics were in units of `f_frsize' rather than > `f_bsize'. But, > since Android's Bionic doesn't have statvfs(3), but, has statfs(2), I > decided to > go with that latter. However, my manpage of statfs(2) doesn't make > clear > whether the units reported are by statfs() are in `f_frsize' or > `f_bsize'. I decided > to go with `f_frsize' given that `f_bsize' is officially the > ``optimal transfer block size''. > > I noticed that both busybox and toybox use `f_bsize', however--which > doesn't > make much difference on most (ie. all those not implementing block > fragments) > filesystems, though. Are there any that implement it? Linus said he wanted to see fragments go away in 1995: http://lkml.indiana.edu/hypermail/linux/kernel/9507/0195.html According to http://www.tldp.org/HOWTO/Partition/appendix.html in ext2, fragment size is hardwired to be equal to block size. So the fact nobody's noticed a field that was deprecated back around Linux 1.2 isn't hooked up in the entire history of uClibc isn't hugely surprising. Rob 1356667654.0 From rajeev_v_pillai at yahoo.com Fri Dec 28 04:52:13 2012 From: rajeev_v_pillai at yahoo.com (Rajeev V. Pillai) Date: Fri, 28 Dec 2012 12:52:13 +0800 (SGT) Subject: [Aboriginal] uClibc-0.9.33.2 statfs() does not populate the `f_frsize' field of `struct statfs' In-Reply-To: <1356667654.31232.16@driftwood> References: <1356642212.92639.YahooMailNeo@web190703.mail.sg3.yahoo.com> (from rajeev_v_pillai@yahoo.com on Thu Dec 27 15:03:32 2012) <1356667654.31232.16@driftwood> Message-ID: <1356670333.7337.YahooMailNeo@web190702.mail.sg3.yahoo.com> > Rob Landley on Friday, December 28, 2012 9:37 AM wrote: > > Are there any that implement it? Linus said he wanted to see fragments go away in 1995: I'm pretty certain that Novell NetWare 4.x natively does it for its filesystem. Not sure if Linux's NWFS implementation handles those fragments, though. Also not sure if FSes which do block suballocation (Btrfs, Reiser4, UFS2) handle the sub-blocks as fragments and report them as such in `f_frsize'. It seems logical that they should. And, given the push towards larger block sizes, more FSes will start to implement something like fragments. 1356670333.0 From rob at landley.net Sat Dec 29 00:45:11 2012 From: rob at landley.net (Rob Landley) Date: Fri, 28 Dec 2012 18:45:11 -0600 Subject: [Aboriginal] uClibc-0.9.33.2 statfs() does not populate the `f_frsize' field of `struct statfs' In-Reply-To: <1356670333.7337.YahooMailNeo@web190702.mail.sg3.yahoo.com> (from rajeev_v_pillai@yahoo.com on Thu Dec 27 22:52:13 2012) Message-ID: <1356741911.31232.18@driftwood> On 12/27/2012 10:52:13 PM, Rajeev V. Pillai wrote: > > Rob Landley on Friday, December 28, 2012 9:37 AM > wrote: > > > > > Are there any that implement it? Linus said he wanted to see > fragments go away in 1995: > > I'm pretty certain that Novell NetWare 4.x natively does it for its > filesystem. > Not sure if Linux's NWFS implementation handles those fragments, > though. > Also not sure if FSes which do block suballocation (Btrfs, Reiser4, > UFS2) > handle the sub-blocks as fragments and report them as such in > `f_frsize'. > It seems logical that they should. I know that reiserfs didn't. And I'm pretty sure btrfs doesn't. The thing is, things like tail packing perform variable sized sub-allocations, so reporting a single "fragment size" number for the whole filesystem is meaningless in that context. (And really, tail packing is a special case.) That fragment field dates back to attempting to have a block size smaller than actual physical transactions were done in, and Linus basically pointed out that the smaller value is the real block size and the larger value is totally artificial so attempting to maintain multiple levels is sad. (The block layer can sort and merge outstanding requests, that's the "I/O elevator" code. Trying to do this is not a filesystem's job.) I.E. _WHEN_ you do block suballocation, the granularity in which you do so is bytes. So the fragment size would always be "1", which is useless. I.E. this really did get discarded 17 years ago and nobody's resurrected it since, because it was a bad idea. > And, given the push towards larger block sizes, more FSes will start > to implement > something like fragments. No, they won't. You're acting like this is a new thing instead of a topic of discussion for many years now: https://lwn.net/Articles/250335/ https://lwn.net/Articles/349970/ Again because a _single_ fragment size is nonsensical, what you want are variable sized chunks. And what you can do to get them is demand that 4096 block ranges be contiguous and then store a count of the number of them you've used, which is an optimization ext2 has been using from day 1 and BSD used before that. If your argument is "we must be able to subdivide filesystem blocks", people do so via byte ranges. (They jump from granularity 4096 to granularity 1.) When it's "we must use larger transactions than filesystem blocks", people group blocks but continue to track the allocation at either block size or byte size. Some media naturally use larger transaction sizes than the filesystem block size, but the fix for that is to make the journaling layer aware of it so it can group the commits. This isn't just a block size issue, it's an alignment issue. When disks started increasing block sizes to _match_ the block sizes filesystems had been using for years, there was a problem that the 512 byte "boot sector" put things out of alignment, and we had to update partitioning programs to create partitions that started at the right offset: https://lwn.net/Articles/322777/ Note that making filesystem block sizes larger than the memory page size didn't happen. And even though we've got _terabytes_ of memory on some of the larger systems, the default RAM allocation size is staying 4k. Yes there are hugepages to conserve TLB entries, and you can format an ext4 filesystem with huge blocks so it doesn't spend forever parsing allocation tables: https://lwn.net/Articles/469821/ But note that when they do that, they don't sub-allocate within hugepages or huge blocks, because doing so DEFEATS THE PURPOSE OF HAVING THEM. This isn't a "fragment size" because you don't fragment them. Subdividing them is the application's problem, not the OS's. Rob 1356741911.0