From patrick at gentoo.org Tue Oct 1 11:16:06 2013 From: patrick at gentoo.org (Patrick Lauer) Date: Tue, 01 Oct 2013 19:16:06 +0800 Subject: [Aboriginal] Load limiter patch Message-ID: <524AAEF6.1090900@gentoo.org> Hi all, I noticed some, ahem, suboptimal behaviour on machines with lots of memory when FORK=1 is used. Problem: the current algorithm sets the number of parallel jobs to memsize/512MB. On a machine with e.g. 32GB that's 64 jobs, since there are 17 targets currently that gives a per-target parallelism of 3 which adds up to 51 parallel jobs ... on a machine with 4 or 8 CPU-cores. You can imagine how that's, err, not happy. So I tried to add a ceiling at 2*CPU, which seems reasonable to me - still allows for good parallelism while not stupidly overwhelming the system. I haven't yet tested it extensively, so there might be some stupid included - but it appears to select the right limit. Enjoy the attached patch, Patrick -------------- next part -------------- A non-text attachment was scrubbed... Name: aboriginal-limit-load.patch Type: text/x-patch Size: 1885 bytes Desc: not available URL: From patrick at gentoo.org Tue Oct 1 11:21:40 2013 From: patrick at gentoo.org (Patrick Lauer) Date: Tue, 01 Oct 2013 19:21:40 +0800 Subject: [Aboriginal] m68k support and qemu-1.6 Message-ID: <524AB044.5000800@gentoo.org> Hi all, I noticed that the available m68k targets apparently changed in qemu-1.6. The current command in sources/targets/m68k is: qemu-system-m68k -M q800 I suggest changing it to: qemu-system-m68k -M an5206 Support is still quite fragile - trying to start the m68k image I get: $ ./run-emulator.sh qemu: fatal: Illegal instruction: 60ff @ 00002000 D0 = 00000000 A0 = 00000000 F0 = 0000000000000000 ( 0) D1 = 00000000 A1 = 00000000 F1 = 0000000000000000 ( 0) D2 = 00000000 A2 = 00000000 F2 = 0000000000000000 ( 0) D3 = 00000000 A3 = 00000000 F3 = 0000000000000000 ( 0) D4 = 00000000 A4 = 00000000 F4 = 0000000000000000 ( 0) D5 = 00000000 A5 = 00000000 F5 = 0000000000000000 ( 0) D6 = 00000000 A6 = 00000000 F6 = 0000000000000000 ( 0) D7 = 00000000 A7 = 00000000 F7 = 0000000000000000 ( 0) PC = 00002000 SR = 2700 ----- FPRESULT = 0 ./run-emulator.sh: line 7: 19280 Aborted qemu-system-m68k -M an5206 -nographic -no-reboot -kernel vmlinux -hda hda.sqf -append "root=/dev/sda rw init=/sbin/init.sh panic=1 PATH=$DISTCC_PATH_PREFIX/bin:/sbin console=ttyAMA0 HOST=m68k $KERNEL_EXTRA" $QEMU_EXTRA 1380626500.0 From patrick at gentoo.org Tue Oct 1 11:25:41 2013 From: patrick at gentoo.org (Patrick Lauer) Date: Tue, 01 Oct 2013 19:25:41 +0800 Subject: [Aboriginal] Patch: fix smoketest timeout behaviour with FORK=1 Message-ID: <524AB135.9010105@gentoo.org> This patch makes the timeout for smoketest configurable and increases the limit when FORK=1 This is needed even on fast machines - on a 3.4Ghz quadcore I get timeouts for sparc, ppc reliably and other targets randomly. With 180sec = 3 minutes it reliably finishes even when all are started in parallel, increasing to 5 minutes or even more might be reasonable. --- ../aboriginal-1.2.5_clone/more/smoketest.sh 2013-09-12 17:21:16.000000000 +0800 +++ more/smoketest.sh 2013-09-23 16:20:15.002910667 +0800 @@ -11,6 +11,8 @@ # If you cat your own script into emulator-build.sh, you probably also need # to start with a line of spaces like that. Just FYI. +$TIMEOUT=${TIMEOUT:-60} + sayhello() { sleep 10 @@ -22,4 +24,4 @@ sleep 1 echo exit } -sayhello | more/timeout.sh 60 more/run-emulator-from-build.sh "$1" +sayhello | more/timeout.sh ${TIMEOUT} more/run-emulator-from-build.sh "$1" --- ../aboriginal-1.2.5_clone/more/smoketest-all.sh 2013-09-12 17:21:16.000000000 +0800 +++ more/smoketest-all.sh 2013-09-23 16:23:10.615505571 +0800 @@ -3,6 +3,10 @@ # Run smoketest script on every build/system-image-* architecture. # If $FORK is set, run them in parallel. +# If $FORK is set increase timeout from default 60sec + +[ -z "$FORK" ] && TIMEOUT=180 + . sources/utility_functions.sh || exit 1 1380626741.0 From patrick at gentoo.org Tue Oct 1 11:34:40 2013 From: patrick at gentoo.org (Patrick Lauer) Date: Tue, 01 Oct 2013 19:34:40 +0800 Subject: [Aboriginal] Failures with qemu-1.6 Message-ID: <524AB350.2020701@gentoo.org> Testing 1.2.5 on amd64 (Gentoo) with qemu 1.6.0 the mips64, sh4, armv6l and sparc targets fail to pass the smoketest. In a way I'm not really surprised how fragile things are, but ... sigh :) armv6l seems to have a problem with the virtual sound hardware in qemu: system-image-armv6l $ ./run-emulator.sh Uncompressing Linux... done, booting the kernel. ALSA lib /chroot/build/portage/media-libs/alsa-lib-1.0.27.2/work/alsa-lib-1.0.27.2/src/pcm/pcm.c:7843:(snd_pcm_recover) underrun occurred sparc seems to have a problem in the compiler: (sparc:1) /home # df Filesystem 1K-blocks Used Available Use% Mounted on /home 54032 0 54032 0% /home /tmp 54032 0 54032 0% /tmp dev 53984 0 53984 0% /dev /dev/root 28416 28416 0 100% / (sparc:1) /home # gcc -s /usr/src/thread-hello2.c -lpthread -o /tmp/hello /tmp/hello exit (sparc:1) /home # /tmp/hello /tmp/hello: symbol '__gcc_personality_v0': can't handle reloc type 0x17 (sparc:1) /home # exit mips64 seems to have invalid executables: 9pnet: Installing 9P2000 support drivers/rtc/hctosys.c: unable to open rtc device (rtc0) VFS: Mounted root (squashfs filesystem) readonly on device 3:0. Freeing YAMON memory: 956k freed Freeing unused kernel memory: 172K (ffffffff80475000 - ffffffff804a0000) Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000a sh4 kernel-oopses at boot: ------------[ cut here ]------------ WARNING: at 8c144a00 [verbose debug info unavailable] CPU: 0 PID: 1 Comm: swapper Not tainted 3.11.0 #1 task: 8fc2c000 ti: 8fc2e000 task.ti: 8fc2e000 PC : 8c144a00 SP : 8fc2fe64 SR : 40008000 TEA : 00000000 R0 : 00000034 R1 : 00000000 R2 : 40008000 R3 : 00000000 R4 : 00000000 R5 : 8c278e04 R6 : 00000002 R7 : 00001aac R8 : 8c255d50 R9 : 8c24b280 R10 : 8fd5c480 R11 : 8c143bf4 R12 : 8fd5c4fc R13 : 00000000 R14 : 00000000 MACH: 00000000 MACL: 0000005b GBR : 00000000 PR : 8c144a00 Call trace: [<8c14d2d0>] 0x8c14d2d0 [<8c0125e0>] 0x8c0125e0 [<8c146d0a>] 0x8c146d0a [<8c0125e0>] 0x8c0125e0 [<8c147060>] 0x8c147060 [<8c14701c>] 0x8c14701c [<8c145a78>] 0x8c145a78 [<8c145834>] 0x8c145834 [<8c1463fc>] 0x8c1463fc [<8c147338>] 0x8c147338 [<8c268094>] 0x8c268094 [<8c26809e>] 0x8c26809e [<8c268094>] 0x8c268094 [<8c25d14e>] 0x8c25d14e [<8c0adc60>] 0x8c0adc60 [<8c032e2a>] 0x8c032e2a [<8c25d5a4>] 0x8c25d5a4 [<8c25d0c0>] 0x8c25d0c0 [<8c25d5b4>] 0x8c25d5b4 [<8c25d0c0>] 0x8c25d0c0 [<8c1f4c46>] 0x8c1f4c46 [<8c0125e0>] 0x8c0125e0 [<8c03b3ac>] 0x8c03b3ac [<8c0171e8>] 0x8c0171e8 [<8c03b3ac>] 0x8c03b3ac [<8c1f4c3c>] 0x8c1f4c3c ---[ end trace 3132d0e6181e7e96 ]--- Uniform Multi-Platform E-IDE driver and later just fails to start init: Type exit when done. (sh4:1) /home # sd 0:0:0:0: [sda] Synchronizing SCSI cache reboot: Restarting system Unauthorized access qemu: fatal: Trying to execute code outside RAM or ROM at 0xa0000000 pc=0xa0000000 sr=0x700000f0 pr=0x8c01331c fpscr=0x00080000 spc=0x8c013322 ssr=0x10000000 gbr=0x00462448 vbr=0x8c018000 sgr=0x8fe75e88 dbr=0x00000000 delayed_pc=0x8c01331c fpul=0x00000000 r0=0x400080f1 r1=0x80000001 r2=0x10000000 r3=0x00000000 r4=0x000000f0 r5=0x8c278e04 r6=0x00000006 r7=0x00002cd8 r8=0x01234567 r9=0x00000000 r10=0xfee1dead r11=0x01234567 r12=0x00000000 r13=0x00000000 r14=0x7bf3bd04 r15=0x8fe75e88 r16=0x00000000 r17=0xffffff0f r18=0x40008000 r19=0x40008000 r20=0x8fe75d2c r21=0x00000000 r22=0x00000000 r23=0x8fe74000 ./run-emulator.sh: line 7: 24407 Aborted qemu-system-sh4 -M r2d -nographic -no-reboot -kernel zImage -hda hda.sqf -append "root=/dev/sda rw init=/sbin/init.sh panic=1 PATH=$DISTCC_PATH_PREFIX/bin:/sbin console=ttySC1 noiotrap HOST=sh4 $KERNEL_EXTRA" $QEMU_EXTRA -monitor null -serial null -serial stdio 1380627280.0 From rob at landley.net Tue Oct 1 15:44:13 2013 From: rob at landley.net (Rob Landley) Date: Tue, 01 Oct 2013 10:44:13 -0500 Subject: [Aboriginal] m68k support and qemu-1.6 In-Reply-To: <524AB044.5000800@gentoo.org> (from patrick@gentoo.org on Tue Oct 1 06:21:40 2013) Message-ID: <1380642253.1974.120@driftwood> On 10/01/2013 06:21:40 AM, Patrick Lauer wrote: > Hi all, > > I noticed that the available m68k targets apparently changed in > qemu-1.6. > > The current command in sources/targets/m68k > is: > qemu-system-m68k -M q800 That's for Laurent Vivier's m68k branch adding support for the last m68k macintosh. Unfortunately, it's been half-finished forever. Last I checked the MMU part of CPU support wasn't there, and the kernel worked up until it tried to enable the page tables, then paniced. > I suggest changing it to: > qemu-system-m68k -M an5206 That's a coldfire board: http://www.uclinux.org/ports/coldfire/binary.html Coldfire is a nommu subset of m68k. Can you plug a real 68030 into it? > Support is still quite fragile - trying to start the m68k image I get: > > $ ./run-emulator.sh > qemu: fatal: Illegal instruction: 60ff @ 00002000 Because coldfire isn't full m68k. The image partially works with the q800 branch of git://gitorious.org/qemu-m68k/qemu-m68k but full support was never there in qemu. (There's another emulator, aranym, that emulates atari boards. The aboriginal userspace was made to work under that, with a different kernel .config, but it doesn't provide all the I/O devices we needed to do an automated linux from scratch build with the current setup. We could presumably do one by hand...) Rob 1380642253.0 From rob at landley.net Thu Oct 3 20:06:09 2013 From: rob at landley.net (Rob Landley) Date: Thu, 03 Oct 2013 15:06:09 -0500 Subject: [Aboriginal] [PATCH] sources/functions.sh: Fix ext2/ext3 build failures In-Reply-To: <1379827815-10151-1-git-send-email-linux@roeck-us.net> (from linux@roeck-us.net on Sun Sep 22 00:30:15 2013) Message-ID: <1380830769.1974.141@driftwood> On 09/22/2013 12:30:15 AM, Guenter Roeck wrote: > Signed-off-by: Guenter Roeck > --- > This patch should apply on top of the 1.2.5 release. It was generated > using git format-patch from a git repo, so hg commands might not work. Sorry I fell so behidn on email, I spent the last week and change job hunting, which I find stressful enough I mostly just played skyrim the rest of the time. Catching up on email now. > sources/functions.sh | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) Applied, although heads up about me wanting to BSD license the aboriginal build scripts. (Still triaging the commit history.) Thanks, Rob -------------- next part -------------- A non-text attachment was scrubbed... Name: LICENSE Type: text/x-patch Size: 1567 bytes Desc: not available URL: From rob at landley.net Thu Oct 3 20:04:55 2013 From: rob at landley.net (Rob Landley) Date: Thu, 03 Oct 2013 15:04:55 -0500 Subject: [Aboriginal] Proposed new LICENSE file. Message-ID: <1380830695.1974.140@driftwood> A non-text attachment was scrubbed... Name: LICENSE Type: text/x-patch Size: 1567 bytes Desc: not available URL: From onefang at gmail.com Fri Oct 4 03:04:44 2013 From: onefang at gmail.com (David Seikel) Date: Fri, 4 Oct 2013 13:04:44 +1000 Subject: [Aboriginal] Proposed new LICENSE file. In-Reply-To: <1380830695.1974.140@driftwood> References: <1380830695.1974.140@driftwood> Message-ID: <20131004130444.657c3d59.onefang@gmail.com> Works for me. So this is my official agreement to have whatever bits I have contributed licensed and that llicense. -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From alessioigorbogani at gmail.com Fri Oct 4 06:04:55 2013 From: alessioigorbogani at gmail.com (Alessio Igor Bogani) Date: Fri, 4 Oct 2013 08:04:55 +0200 Subject: [Aboriginal] Proposed new LICENSE file. In-Reply-To: <20131004130444.657c3d59.onefang@gmail.com> References: <1380830695.1974.140@driftwood> <20131004130444.657c3d59.onefang@gmail.com> Message-ID: On 4 October 2013 05:04, David Seikel wrote: > Works for me. So this is my official agreement to have whatever bits I > have contributed licensed and that llicense. It is same for me. Ciao, Alessio 1380866695.0 From linux at roeck-us.net Thu Oct 3 22:09:06 2013 From: linux at roeck-us.net (Guenter Roeck) Date: Thu, 03 Oct 2013 15:09:06 -0700 Subject: [Aboriginal] [PATCH] sources/functions.sh: Fix ext2/ext3 build failures In-Reply-To: <1380830769.1974.141@driftwood> References: <1380830769.1974.141@driftwood> Message-ID: <524DEB02.8050609@roeck-us.net> On 10/03/2013 01:06 PM, Rob Landley wrote: > On 09/22/2013 12:30:15 AM, Guenter Roeck wrote: >> Signed-off-by: Guenter Roeck >> --- >> This patch should apply on top of the 1.2.5 release. It was generated >> using git format-patch from a git repo, so hg commands might not work. > > Sorry I fell so behidn on email, I spent the last week and change job hunting, which I find stressful enough I mostly just played skyrim the rest of the time. Catching up on email now. > >> sources/functions.sh | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) > > Applied, although heads up about me wanting to BSD license the aboriginal build scripts. (Still triaging the commit history.) > Fine with me - I don't put any license limitations or restrictions on patches like that. Cheers, and good luck! Guenter 1380838146.0 From rob at landley.net Fri Oct 4 19:55:56 2013 From: rob at landley.net (Rob Landley) Date: Fri, 04 Oct 2013 14:55:56 -0500 Subject: [Aboriginal] bootstrapping llvm In-Reply-To: <5B7A395C-7FB7-4CAB-B2E6-7B82FE1C1165@securivm.ca> (from alain.toussaint@securivm.ca on Wed Sep 25 14:45:03 2013) References: <9B3BF685-CB5E-4CB3-A41E-65660EA86AAE@securivm.ca> <1380121860.1974.83@driftwood> <5B7A395C-7FB7-4CAB-B2E6-7B82FE1C1165@securivm.ca> Message-ID: <1380916556.1974.155@driftwood> On 09/25/2013 02:45:03 PM, Alain Toussaint wrote: > At the moment, I have built llvm 3.3 under gentoo with all the > targets enabled > but tonight when I get back home, I'll try to build a cross-compiler > in my homedir. I note my record-command.sh and report-recorded-commands.sh scripts (see the FAQ) are useful work working out build prerequisites. They're available in the build on the host, and get installed into the system images as a command line wrapper. Rob 1380916556.0 From alain.toussaint at securivm.ca Sat Oct 5 03:19:34 2013 From: alain.toussaint at securivm.ca (Alain Toussaint) Date: Fri, 4 Oct 2013 23:19:34 -0400 Subject: [Aboriginal] bootstrapping llvm In-Reply-To: <1380916556.1974.155@driftwood> References: <9B3BF685-CB5E-4CB3-A41E-65660EA86AAE@securivm.ca> <1380121860.1974.83@driftwood> <5B7A395C-7FB7-4CAB-B2E6-7B82FE1C1165@securivm.ca> <1380916556.1974.155@driftwood> Message-ID: <053A64F9-0295-4693-A54B-C9EB570B0B1B@securivm.ca> > On 09/25/2013 02:45:03 PM, Alain Toussaint wrote: >> At the moment, I have built llvm 3.3 under gentoo with all the targets enabled >> but tonight when I get back home, I'll try to build a cross-compiler in my homedir. > > I note my record-command.sh and report-recorded-commands.sh scripts (see the FAQ) are useful work working out build prerequisites. They're available in the build on the host, and get installed into the system images as a command line wrapper. > > Rob Ok, I'll look into their sources. For the moment, I've been looking at building llvm with musl using this page: http://wiki.musl-libc.org/wiki/BuildingLLVM I don't know yet if I'll follow the howto to the letter or adapt it to use a llvm userland (the instruction refer to a libgcc & libstdc++ build and I actually want to build with their clang/llvm counterpart) but I will decide later next week. Alain -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob at landley.net Sat Oct 5 21:27:31 2013 From: rob at landley.net (Rob Landley) Date: Sat, 05 Oct 2013 16:27:31 -0500 Subject: [Aboriginal] bootstrapping llvm In-Reply-To: <053A64F9-0295-4693-A54B-C9EB570B0B1B@securivm.ca> (from alain.toussaint@securivm.ca on Fri Oct 4 22:19:34 2013) Message-ID: <1381008451.1974.160@driftwood> On 10/04/2013 10:19:34 PM, Alain Toussaint wrote: > > On 09/25/2013 02:45:03 PM, Alain Toussaint wrote: > >> At the moment, I have built llvm 3.3 under gentoo with all the > targets enabled > >> but tonight when I get back home, I'll try to build a > cross-compiler in my homedir. > > > > I note my record-command.sh and report-recorded-commands.sh scripts > (see the FAQ) are useful work working out build prerequisites. > They're available in the build on the host, and get installed into > the system images as a command line wrapper. > > > > Rob > > Ok, I'll look into their sources. For the moment, I've been looking > at building > llvm with musl using this page: > http://wiki.musl-libc.org/wiki/BuildingLLVM Cool. > I don't know yet if I'll follow the howto to the letter or adapt it > to use a llvm > userland (the instruction refer to a libgcc & libstdc++ build and I > actually want > to build with their clang/llvm counterpart) but I will decide later > next week. The history of llvm involves creating a new code generation backend using the gcc frontend (llvm), then creating a new frontend (clang), then creating new libgcc and libstdc++, then creating a new assembler and linker, then fixing various packages that had upstream assumptions... It's always been an incremental process to get to the desired result. I usually try to get something working then swap out parts one at a time with a known-good base to compare against. Once I get a knew known-good base, archive it and iterate with the next change. Often the first "get something working" is really crappy, from just "build it on my ubuntu host with their gcc and glibc and $PATH of tools" through "boot somebody else's image under qemu and poke around". If you've played with Linux From Scratch, you'll see where I got the "populate a chroot with tools, then build another chroot using _that_ chroot" approach. It's an excellent way to encapsulate your dependencies; if the first chroot is missing something it can't build the second. I'd make it work on x86 with just chroot, and _then_ worry about other targets and qemu. (You could even do actual linux from scratch using the gnu tools and then worry about swapping them out for busybox after you have a known working baseline. I doubt the llvm guys have been testing busybox/toybox as their build environment. :) Rob 1381008451.0 From rob at landley.net Sun Oct 6 07:38:03 2013 From: rob at landley.net (Rob Landley) Date: Sun, 06 Oct 2013 02:38:03 -0500 Subject: [Aboriginal] release with kernel 3.11 and posix_spawn ? In-Reply-To: <524482BD.7090209@barfooze.de> (from maillist-aboriginal@barfooze.de on Thu Sep 26 13:53:49 2013) References: <524482BD.7090209@barfooze.de> Message-ID: <1381045083.1974.169@driftwood> On 09/26/2013 01:53:49 PM, John Spencer wrote: > did i miss the release announcement, or is the 3.11 release still in > the works? Ah, sorry. It's on the website, but I didn't ping the mailing list. (I'm still about 6000 emails behind. Down from just under 10,000 earlier this week. When I got home from Ohio Linuxfest I start job hunting in earnest, and that stressed me out enough I spent the rest of the time playing skyrim and avoiding my computer. Accepted an offer on wednesday and got all the paperwork in thursday, slowly catching up since.) http://landley.net/aboriginal/ has the release announcement. I think the correct binaries are all up? > i could make good use of it currently as i plan to use it for another > sabotage > release for all supported archs, but this time without posix_spawn > headaches... Let me know if anything needs tweaking. It should work, I think... Rob 1381045083.0 From maillist-aboriginal at barfooze.de Sun Oct 6 08:31:05 2013 From: maillist-aboriginal at barfooze.de (John Spencer) Date: Sun, 06 Oct 2013 10:31:05 +0200 Subject: [Aboriginal] release with kernel 3.11 and posix_spawn ? In-Reply-To: <1381045083.1974.169@driftwood> References: <524482BD.7090209@barfooze.de> <1381045083.1974.169@driftwood> Message-ID: <52511FC9.7040507@barfooze.de> On 10/06/2013 09:38 AM, Rob Landley wrote: > On 09/26/2013 01:53:49 PM, John Spencer wrote: >> did i miss the release announcement, or is the 3.11 release still in >> the works? > > Ah, sorry. It's on the website, but I didn't ping the mailing list. (I'm I see. it appears your hg repo is also missing a release tag. > still about 6000 emails behind. Down from just under 10,000 earlier this > week. When I got home from Ohio Linuxfest I start job hunting in > earnest, and that stressed me out enough I spent the rest of the time > playing skyrim and avoiding my computer. Accepted an offer on wednesday > and got all the paperwork in thursday, slowly catching up since.) > > http://landley.net/aboriginal/ has the release announcement. I think the > correct binaries are all up? > >> i could make good use of it currently as i plan to use it for another >> sabotage >> release for all supported archs, but this time without posix_spawn >> headaches... > > Let me know if anything needs tweaking. It should work, I think... Let me try... > > Rob 1381048265.0 From rob at landley.net Sun Oct 6 19:32:53 2013 From: rob at landley.net (Rob Landley) Date: Sun, 06 Oct 2013 14:32:53 -0500 Subject: [Aboriginal] release with kernel 3.11 and posix_spawn ? In-Reply-To: <52511FC9.7040507@barfooze.de> (from maillist-aboriginal@barfooze.de on Sun Oct 6 03:31:05 2013) References: <524482BD.7090209@barfooze.de> <1381045083.1974.169@driftwood> <52511FC9.7040507@barfooze.de> Message-ID: <1381087973.1974.173@driftwood> On 10/06/2013 03:31:05 AM, John Spencer wrote: > On 10/06/2013 09:38 AM, Rob Landley wrote: >> On 09/26/2013 01:53:49 PM, John Spencer wrote: >>> did i miss the release announcement, or is the 3.11 release still in >>> the works? >> >> Ah, sorry. It's on the website, but I didn't ping the mailing list. >> (I'm > > I see. it appears your hg repo is also missing a release tag. Oops. Fixed. Sorry, off poking at the LFS 7.4 migration when I've been doing aboriginal stuff. I decided package version upgrades were a good idea before trying to make musl work, because fixing stuff in obsolete package versions would be a waste of time... But mostly, really behind on everything... Rob 1381087973.0 From rob at landley.net Wed Oct 16 01:53:17 2013 From: rob at landley.net (Rob Landley) Date: Tue, 15 Oct 2013 20:53:17 -0500 Subject: [Aboriginal] Load limiter patch In-Reply-To: <524AAEF6.1090900@gentoo.org> (from patrick@gentoo.org on Tue Oct 1 06:16:06 2013) Message-ID: <1381888399.1974.197@driftwood> On 10/01/2013 06:16:06 AM, Patrick Lauer wrote: > Hi all, Hello! Massively behind on email, sorry about that. Working through the backlog... > I noticed some, ahem, suboptimal behaviour on machines with lots of > memory when FORK=1 is used. Used to do... more/buildall.sh? > Problem: the current algorithm sets the number of parallel jobs to > memsize/512MB. On a machine with e.g. 32GB that's 64 jobs, since there > are 17 targets currently that gives a per-target parallelism of 3 > which > adds up to 51 parallel jobs ... on a machine with 4 or 8 CPU-cores. > > You can imagine how that's, err, not happy. The limit was really there to avoid swap-thrashing, not to optimize CPU utilization. Some portions of the build are serialized, such as tarball creation and building the ancient bash package that fails with -j higher than 1. Anything doing link time optimization is also going to grind through very fast "compilation" stages that are barely more than the "gcc -E" preprocessing distcc does, and then do all the actual work when he linker calls a plugin back to the compiler to do the actual code generation on the whole tree at once: this deferred code generation is unfortunately a single processor operation. The FSF loonies did this rather than rewrite all the makefiles out there to feed all the *.c files to the compiler at once instead of calling the compiler a zillion times with one file at a time, and then the linker was the only thing that actually saw _everything_, so if they wanted the optimizer to see the whole program in one go they had to defer the actual code generation pass to have the linker do it, or admit "autoconf/automake/gmake" is a deeply broken paradigm. (Never admit mistakes and backtrack, you must DOUBLE DOWN. That's the FSF way.) > So I tried to add a ceiling at 2*CPU, which seems reasonable to me - > still allows for good parallelism while not stupidly overwhelming the > system. I haven't yet tested it extensively, so there might be some > stupid included - but it appears to select the right limit. > > Enjoy the attached patch, Does this take into account what the rest of the system is doing? I.E. if buildall.sh fires off a dozen build.sh instances in parallel, they'll work it out among themselves automatically? (I trust gnu code about as far as I can throw it, and thus haven't really told make to handle this for me, but if it actually works...) Rob 1381888397.0 From rob at landley.net Wed Oct 16 02:07:47 2013 From: rob at landley.net (Rob Landley) Date: Tue, 15 Oct 2013 21:07:47 -0500 Subject: [Aboriginal] Load limiter patch In-Reply-To: <524AAEF6.1090900@gentoo.org> (from patrick@gentoo.org on Tue Oct 1 06:16:06 2013) Message-ID: <1381889267.1974.198@driftwood> On 10/01/2013 06:16:06 AM, Patrick Lauer wrote: > Hi all, > > I noticed some, ahem, suboptimal behaviour on machines with lots of > memory when FORK=1 is used. > > Problem: the current algorithm sets the number of parallel jobs to > memsize/512MB. On a machine with e.g. 32GB that's 64 jobs, since there > are 17 targets currently that gives a per-target parallelism of 3 > which > adds up to 51 parallel jobs ... on a machine with 4 or 8 CPU-cores. > > You can imagine how that's, err, not happy. Ok, now that I've read your actual patch, let me write a different response. :) When you said "load limiter" I thought you were using the "-l LOAD" option of make, I.E. --load-average. But you're just capping CPUs, and the way you're doing it is an integer division: CPUS=$(( $REAL_CPUS / $TARGET_COUNT )) Means that if REAL_CPUS is 8 and TARGET_COUNT is 12, you get 0 (so each target basically gets $REAL_CPUS). Or if REAL_CPUS is 16 and TARGET_COUNT is 12 you get 1 and leave 4 processors idle... That's why I didn't do that; integer rounding gets ugly no matter how you look at it. It's possible that telling make to do its own load balancing will work out better. I honestly don't know, see "don't trust anything the FSF implements to not be broken", but that looks like the thing to investigate? I also note that what I tend to want is to max out CPUS for the host-tools and simple-cross-compiler i686 builds, and then CPUS=1 for all the other target builds so I get a coherent log out of them. (Building with CPUS=1 means the log of the build output is harder to read if you care about that sort of thing.) That still gives pretty reasonable parallelism on the systems I'm building on, but shouldn't be the default because if you have a 64 processor monster it won't take advantage of it at all. I've been meaning to do a knob for that, but haven't yet. I generally just "CPUS=1 FORK=1 more/buildall.sh" to do release builds... Sigh. And these days, FORK=1 should probably be the default and I should change the variable to NOFORK=1 to switch it off... Rob 1381889267.0 From rob at landley.net Wed Oct 16 02:41:27 2013 From: rob at landley.net (Rob Landley) Date: Tue, 15 Oct 2013 21:41:27 -0500 Subject: [Aboriginal] m68k support and qemu-1.6 In-Reply-To: <524AB044.5000800@gentoo.org> (from patrick@gentoo.org on Tue Oct 1 06:21:40 2013) References: <524AB044.5000800@gentoo.org> Message-ID: <1381891287.1974.199@driftwood> On 10/01/2013 06:21:40 AM, Patrick Lauer wrote: > Hi all, > > I noticed that the available m68k targets apparently changed in > qemu-1.6. > > The current command in sources/targets/m68k > is: > qemu-system-m68k -M q800 Did I already reply to this one? About Laurient Vivier's m68k branch and all? Think I did... Yeah, I see it in the archive. I note that I recently suggested Laurent become qemu-m68k maintainer, and he was up for it: http://lists.nongnu.org/archive/html/qemu-devel/2013-10/msg00386.html He'd be the logical guy to follow up with on that. (It's on my todo list, but you can see from my email how well I'm doing keeping up with that, and my new day job starts monday.) Rob 1381891287.0 From rob at landley.net Wed Oct 16 02:55:50 2013 From: rob at landley.net (Rob Landley) Date: Tue, 15 Oct 2013 21:55:50 -0500 Subject: [Aboriginal] Patch: fix smoketest timeout behaviour with FORK=1 In-Reply-To: <524AB135.9010105@gentoo.org> (from patrick@gentoo.org on Tue Oct 1 06:25:41 2013) Message-ID: <1381892150.1974.200@driftwood> On 10/01/2013 06:25:41 AM, Patrick Lauer wrote: > This patch makes the timeout for smoketest configurable and increases > the limit when FORK=1 The native-build.sh timeout was already configurable, and the problem with smoketest.sh is that stdin eats an unknown amount of data during initialization. (It varies by target; serial chip initialization flushes the buffer and stuff like powerpc with a 1024 character buffer....) So the "cat thingy | qemu" case doesn't work because it may eat all of the input before the shell starts.) I need something more like expect where it waits for a prompt THEN sends data. (I started writing a shell script that made FIFOs, and got it working, but it was too complex for me to be comfortable with.) The alternative is feeding it a hello world build control image, which I've done but that doesn't work for stuff like sh4 that can't give me a third drive to attach the image to. Longer term, I'm poking at having the base system (toybox+musl, enough to boot to a shell prompt) live in initramfs, and then: 1) Putting network block device client and server in toybox. (I did the client for busybox, but need to write the server side.) That lets us mount the native compiler (hda squashfs image) without an emulated block device, and also the build control image. 2) Putting a 9p server in toybox, and letting the virtual system mount a directory from the host to use as /home (hdb ext2 image). (Posibly the 9p server could also do the hda and hdc images, but that's a farish todo item.) > This is needed even on fast machines - on a 3.4Ghz quadcore I get > timeouts for sparc, ppc reliably and other targets randomly. Sparc qemu is unstable. It'll hang if the host running the emulator is doing much else. Some sort of timing issue. Dunno if it's the kernel or the emulator. > With 180sec = 3 minutes it reliably finishes even when all are started > in parallel, increasing to 5 minutes or even more might be reasonable. Indeed. I'm applying this patch, but long-term smoketest.sh needs a rewrite along the lines mentioned above. Rob 1381892150.0 From rob at landley.net Wed Oct 16 03:52:52 2013 From: rob at landley.net (Rob Landley) Date: Tue, 15 Oct 2013 22:52:52 -0500 Subject: [Aboriginal] Failures with qemu-1.6 In-Reply-To: <524AB350.2020701@gentoo.org> (from patrick@gentoo.org on Tue Oct 1 06:34:40 2013) Message-ID: <1381895572.1974.201@driftwood> On 10/01/2013 06:34:40 AM, Patrick Lauer wrote: > Testing 1.2.5 on amd64 (Gentoo) with qemu 1.6.0 the mips64, sh4, > armv6l > and sparc targets fail to pass the smoketest. mips64: there's more than one type of 64 bit mips, and the one I implemented was obsolete and has bit-rotted badly. I never got it 100% working in the first place because when i got the kernel booted, it was broken: http://landley.net/notes-2011.html#17-09-2011 Once upon a time there were 64 bit mips workstations something like a decade ago, and the company making them stopped doing it. (Might have been silicon graphics? I don't remember.) More recently they've reintroduced 64 bit support to mips but did it in a different way, and I haven't done the research to see how to configure that. (It's a bit like the mips-r4k vs "mips32" thing. the old r4k stuff is limited to 256 megs physical memory, the mips32 stuff isn't, but I haven't done the research on a kernel config and board emulation and so on. So many weird little variants of each architecture...) Fixing mips64 again is on my todo list, but what I really want to do is switch it to the _current_ variant (one people might actually _use_) and I don't have an example of that. Need to dig through debian and such to see if they have an image I can get qemu to run... sh4: Two problems: 1) the serial port's broken in qemu, my patch for that _finally_ went upstream but wasn't in 1.6.0. (It might be in 1.6.1?) http://lists.nongnu.org/archive/html/qemu-devel/2013-10/msg00628.html 2) The board qemu emulates for sh4 only has 64 megs physical memory (hardwired), and modern gcc isn't particularly happy with less than 256 megs. It can just about build hello world and thus pass the smoketest, but if you try to do the linux from scratch build it won't make it very far. (I need to hack up qemu and the kernel to add another 196 megs of physical memory via the discontigmem stuff. It's on the todo list...) Until then, even when it's working it isn't very useful. armv6l: um, works for me here? Ah, I have 1.5.1 in the $PATH. So the system image is working fine, this sounds like a qemu problem. I'll add it to the heap... > In a way I'm not really surprised how fragile things are, but ... > sigh :) Everything breaks every release. Alas, you're seeing targets I didn't put enough effort into before release. (Sparc32 isn't properly supported by uClibc, the long term fix is to switch to musl and make sparc work properly there. I only got it working for dynamic linking at _all_ fairly recently, and then it broke again...) sh4 isn't quite properly supported by qemu (insufficient memory in board emulation). armv6l is redundant for my purposes (armv5l is the pentium of arm, armv6l mostly added SMP support which qemu doesn't emulate and which doesn't affect userspace anyway; armv7l is the next big step), and mips64 never worked right in the first place and needs a redo. But you're right, I should fix all of 'em. I've gotten a bit lax lately, so much else to do... > armv6l seems to have a problem with the virtual sound hardware in > qemu: > > system-image-armv6l $ ./run-emulator.sh > Uncompressing Linux... done, booting the kernel. > ALSA lib > /chroot/build/portage/media-libs/alsa-lib-1.0.27.2/work/alsa-lib-1.0.27.2/src/pcm/pcm.c:7843:(snd_pcm_recover) > underrun occurred > > sparc seems to have a problem in the compiler: > > (sparc:1) /home # df > Filesystem 1K-blocks Used Available Use% Mounted on > /home 54032 0 54032 0% /home > /tmp 54032 0 54032 0% /tmp > dev 53984 0 53984 0% /dev > /dev/root 28416 28416 0 100% / > (sparc:1) /home # gcc -s /usr/src/thread-hello2.c -lpthread -o > /tmp/hello > /tmp/hello > exit > (sparc:1) /home # /tmp/hello > > /tmp/hello: symbol '__gcc_personality_v0': can't handle reloc type > 0x17 > (sparc:1) /home # exit I think it's actually in libc, but yeah. I had it working at one point. Grr... > mips64 seems to have invalid executables: > > 9pnet: Installing 9P2000 support > drivers/rtc/hctosys.c: unable to open rtc device (rtc0) > VFS: Mounted root (squashfs filesystem) readonly on device 3:0. > Freeing YAMON memory: 956k freed > Freeing unused kernel memory: 172K (ffffffff80475000 - > ffffffff804a0000) > Kernel panic - not syncing: Attempted to kill init! > exitcode=0x0000000a I need to bisect and see which package upgrade broke it. (Either the kernel .config or uClibc, almost certainly...) > sh4 kernel-oopses at boot: > > ------------[ cut here ]------------ > WARNING: at 8c144a00 [verbose debug info unavailable] > > CPU: 0 PID: 1 Comm: swapper Not tainted 3.11.0 #1 Warning? Warning _what_? What's the error message? > task: 8fc2c000 ti: 8fc2e000 task.ti: 8fc2e000 > PC : 8c144a00 SP : 8fc2fe64 SR : 40008000 TEA : 00000000 > R0 : 00000034 R1 : 00000000 R2 : 40008000 R3 : 00000000 > R4 : 00000000 R5 : 8c278e04 R6 : 00000002 R7 : 00001aac > R8 : 8c255d50 R9 : 8c24b280 R10 : 8fd5c480 R11 : 8c143bf4 > R12 : 8fd5c4fc R13 : 00000000 R14 : 00000000 > MACH: 00000000 MACL: 0000005b GBR : 00000000 PR : 8c144a00 > > Call trace: > [<8c14d2d0>] 0x8c14d2d0 > [<8c0125e0>] 0x8c0125e0 > [<8c146d0a>] 0x8c146d0a > [<8c0125e0>] 0x8c0125e0 > [<8c147060>] 0x8c147060 > [<8c14701c>] 0x8c14701c > [<8c145a78>] 0x8c145a78 > [<8c145834>] 0x8c145834 > [<8c1463fc>] 0x8c1463fc > [<8c147338>] 0x8c147338 > [<8c268094>] 0x8c268094 > [<8c26809e>] 0x8c26809e > [<8c268094>] 0x8c268094 > [<8c25d14e>] 0x8c25d14e > [<8c0adc60>] 0x8c0adc60 > [<8c032e2a>] 0x8c032e2a > [<8c25d5a4>] 0x8c25d5a4 > [<8c25d0c0>] 0x8c25d0c0 > [<8c25d5b4>] 0x8c25d5b4 > [<8c25d0c0>] 0x8c25d0c0 > [<8c1f4c46>] 0x8c1f4c46 > [<8c0125e0>] 0x8c0125e0 > [<8c03b3ac>] 0x8c03b3ac > [<8c0171e8>] 0x8c0171e8 > [<8c03b3ac>] 0x8c03b3ac > [<8c1f4c3c>] 0x8c1f4c3c > > ---[ end trace 3132d0e6181e7e96 ]--- > Uniform Multi-Platform E-IDE driver > > and later just fails to start init: > > Type exit when done. > (sh4:1) /home # sd 0:0:0:0: [sda] Synchronizing SCSI cache > reboot: Restarting system So you get a shell prompt, and the shell prompt gets EOF. Is this the interactive behavior, or is this smoketest.sh dying because serial init consumed all input resetting the emulated chip and discarding its buffered I/O? See "smoketest needs a rewrite". I note that for the script that generates the screenshots page (which I don't think I remembered to run for the most recent releae, actually), it does a horrible thing like: (sleep 10; echo $PARTONE; sleep 5; echo $PARTTWO) | qemu The _sad_ part is with a couple targets, if you feed it too much data it discards everything that won't fit in the buffer. So you have to chop the input into small enough chunks. Again, the control-image stuff is more reliable than scripting the emulator from outside. When you have a /dev/hdc, of course... > Unauthorized access > qemu: fatal: Trying to execute code outside RAM or ROM at 0xa0000000 Sigh. that part's normal, sh4 doesn't reset properly and thus panics on the way out. Dunno if that's kernel or qemu, I'm guessing qemu. (It tries to jump to a ROM to perform the reset, and the ROM isn't there.) Either way it exits, so it's a purely cosmetic issue. Rob 1381895572.0 From patrick at gentoo.org Wed Oct 16 06:16:58 2013 From: patrick at gentoo.org (Patrick Lauer) Date: Wed, 16 Oct 2013 14:16:58 +0800 Subject: [Aboriginal] Load limiter patch In-Reply-To: <1381889267.1974.198@driftwood> References: <1381889267.1974.198@driftwood> Message-ID: <525E2F5A.6080607@gentoo.org> On 10/16/2013 10:07 AM, Rob Landley wrote: > On 10/01/2013 06:16:06 AM, Patrick Lauer wrote: >> Hi all, >> >> I noticed some, ahem, suboptimal behaviour on machines with lots of >> memory when FORK=1 is used. >> >> Problem: the current algorithm sets the number of parallel jobs to >> memsize/512MB. On a machine with e.g. 32GB that's 64 jobs, since there >> are 17 targets currently that gives a per-target parallelism of 3 which >> adds up to 51 parallel jobs ... on a machine with 4 or 8 CPU-cores. >> >> You can imagine how that's, err, not happy. > > Ok, now that I've read your actual patch, let me write a different > response. :) > > When you said "load limiter" I thought you were using the "-l LOAD" > option of make, I.E. --load-average. But you're just capping CPUs, and > the way you're doing it is an integer division: > > CPUS=$(( $REAL_CPUS / $TARGET_COUNT )) > > Means that if REAL_CPUS is 8 and TARGET_COUNT is 12, you get 0 (so each > target basically gets $REAL_CPUS). Or if REAL_CPUS is 16 and > TARGET_COUNT is 12 you get 1 and leave 4 processors idle... Yeah. You could add a fudge factor there so it overshoots a bit more ... without limiting you're going to run a local DoS ;) For my usecases it seems to work out quite well with a maximum of 2*REALCPU as upper limit, so on a 16-core box you'd limit to 32. Right now worst case you end up with 17 targets * N jobs paralleism per target, which is 17*3 with 32G or 17*6 with 64G. Unless you add nice+ionice by default that'll be very unpleasant :) (Actually - I started working on this patch because I got some spurious performance warnings from my server, which accidentally correlate with the times when you were logged in. So, I guess, maybe limiting resource use would be more awesome. Not that it's really causing problems ... just annoying) Have fun, Patrick 1381904218.0 From vda.linux at googlemail.com Thu Oct 17 02:31:19 2013 From: vda.linux at googlemail.com (Denys Vlasenko) Date: Thu, 17 Oct 2013 04:31:19 +0200 Subject: [Aboriginal] about the strace-android In-Reply-To: References: Message-ID: <201310170431.19494.vda.linux@googlemail.com> On Wednesday 02 October 2013 18:03, Xiaoyong Zhou wrote: > Hi, Denys: > > I just forked the strace-android but the project does not seem to have > enough documentation. I would like to use it to trace binder ioctl on > android and seems that this project did some improvements. > > Could you tell me what's the enhancements and how to compile it for android > phone (arm)? You can compile strace for ARM using a cross-compiler or a native compiler for ARM. For example, Aboriginal Linux provides qemu-runnable filesystem images with compiler, kernel and other misc stuff, under which you can build strace. Please read strace/qemu_multiarch_testing/README. 1381977079.0 From eric at laganowski.net Fri Oct 18 21:32:45 2013 From: eric at laganowski.net (Eric Laganowski) Date: Fri, 18 Oct 2013 17:32:45 -0400 Subject: [Aboriginal] toolchain placement options Message-ID: An HTML attachment was scrubbed... URL: From rob at landley.net Sat Oct 19 05:59:31 2013 From: rob at landley.net (Rob Landley) Date: Sat, 19 Oct 2013 00:59:31 -0500 Subject: [Aboriginal] bootstrapping llvm In-Reply-To: <053A64F9-0295-4693-A54B-C9EB570B0B1B@securivm.ca> (from alain.toussaint@securivm.ca on Fri Oct 4 22:19:34 2013) References: <9B3BF685-CB5E-4CB3-A41E-65660EA86AAE@securivm.ca> <1380121860.1974.83@driftwood> <5B7A395C-7FB7-4CAB-B2E6-7B82FE1C1165@securivm.ca> <1380916556.1974.155@driftwood> <053A64F9-0295-4693-A54B-C9EB570B0B1B@securivm.ca> Message-ID: <1382162371.1974.211@driftwood> On 10/04/2013 10:19:34 PM, Alain Toussaint wrote: > > On 09/25/2013 02:45:03 PM, Alain Toussaint wrote: > >> At the moment, I have built llvm 3.3 under gentoo with all the > targets enabled > >> but tonight when I get back home, I'll try to build a > cross-compiler in my homedir. > > > > I note my record-command.sh and report-recorded-commands.sh scripts > (see the FAQ) are useful work working out build prerequisites. > They're available in the build on the host, and get installed into > the system images as a command line wrapper. > > > > Rob > > Ok, I'll look into their sources. For the moment, I've been looking > at building > llvm with musl using this page: > http://wiki.musl-libc.org/wiki/BuildingLLVM Oh, by the way: http://ellcc.org/blog/?p=231 The guy who does that is rdp on the #musl twitter... Rob 1382162371.0 From rob at landley.net Sat Oct 19 06:00:12 2013 From: rob at landley.net (Rob Landley) Date: Sat, 19 Oct 2013 01:00:12 -0500 Subject: [Aboriginal] bootstrapping llvm In-Reply-To: <053A64F9-0295-4693-A54B-C9EB570B0B1B@securivm.ca> (from alain.toussaint@securivm.ca on Fri Oct 4 22:19:34 2013) References: <9B3BF685-CB5E-4CB3-A41E-65660EA86AAE@securivm.ca> <1380121860.1974.83@driftwood> <5B7A395C-7FB7-4CAB-B2E6-7B82FE1C1165@securivm.ca> <1380916556.1974.155@driftwood> <053A64F9-0295-4693-A54B-C9EB570B0B1B@securivm.ca> Message-ID: <1382162412.1974.212@driftwood> On 10/04/2013 10:19:34 PM, Alain Toussaint wrote: > > On 09/25/2013 02:45:03 PM, Alain Toussaint wrote: > >> At the moment, I have built llvm 3.3 under gentoo with all the > targets enabled > >> but tonight when I get back home, I'll try to build a > cross-compiler in my homedir. > > > > I note my record-command.sh and report-recorded-commands.sh scripts > (see the FAQ) are useful work working out build prerequisites. > They're available in the build on the host, and get installed into > the system images as a command line wrapper. > > > > Rob > > Ok, I'll look into their sources. For the moment, I've been looking at > building llvm with musl using this page: > http://wiki.musl-libc.org/wiki/BuildingLLVM I mean he's rdp on the #musl freenode IRC channel. (Juggling too many things these days...) Rob 1382162412.0 From rob at landley.net Tue Oct 29 01:00:15 2013 From: rob at landley.net (Rob Landley) Date: Mon, 28 Oct 2013 20:00:15 -0500 Subject: [Aboriginal] Load limiter patch In-Reply-To: <525E2F5A.6080607@gentoo.org> (from patrick@gentoo.org on Wed Oct 16 01:16:58 2013) Message-ID: <1383008415.1974.228@driftwood> On 10/16/2013 01:16:58 AM, Patrick Lauer wrote: > On 10/16/2013 10:07 AM, Rob Landley wrote: > > When you said "load limiter" I thought you were using the "-l LOAD" > > option of make, I.E. --load-average. But you're just capping CPUs, > and > > the way you're doing it is an integer division: > > > > CPUS=$(( $REAL_CPUS / $TARGET_COUNT )) > > > > Means that if REAL_CPUS is 8 and TARGET_COUNT is 12, you get 0 (so > each > > target basically gets $REAL_CPUS). Or if REAL_CPUS is 16 and > > TARGET_COUNT is 12 you get 1 and leave 4 processors idle... > > Yeah. You could add a fudge factor there so it overshoots a bit more > ... > without limiting you're going to run a local DoS ;) > > For my usecases it seems to work out quite well with a maximum of > 2*REALCPU as upper limit, so on a 16-core box you'd limit to 32. You can manually control it by specifying CPU=2 (it's the per-build value). As always, I need to write more documentation. And then find a way to deal with the documentation (already) being too long for anybody to read. Maybe I should do an instructional video series? My new laptop probably isn't so horribly overwhelmed to be a slideshow with audio dropouts when I try that like the netbook was. :) > Right now worst case you end up with 17 targets * N jobs paralleism > per > target, which is 17*3 with 32G or 17*6 with 64G. > > Unless you add nice+ionice by default that'll be very unpleasant :) I've pondered having buildall.sh automatically do nice -n 20. Doing ionice as well makes sense. > (Actually - I started working on this patch because I got some > spurious > performance warnings from my server, which accidentally correlate with > the times when you were logged in. So, I guess, maybe limiting > resource > use would be more awesome. Not that it's really causing problems ... > just annoying) I got a much more powerful laptop and have mostly been using that. I might set up some cron jobs on the server to do nightly tests of new kernels and stuff, if that's ok. > Have fun, > > Patrick Rob 1383008415.0 From rob at landley.net Mon Oct 28 15:34:55 2013 From: rob at landley.net (Rob Landley) Date: Mon, 28 Oct 2013 10:34:55 -0500 Subject: [Aboriginal] toolchain placement options In-Reply-To: (from eric@laganowski.net on Fri Oct 18 16:32:45 2013) Message-ID: <1382974495.1974.227@driftwood> Sorry for the late answer. Started a new job last monday and I was behind on my email before that... On 10/18/2013 04:32:45 PM, Eric Laganowski wrote: > Hello, > > First I want to thank everyone involved in the Aboriginal Linux for > their hard work implementing something that it seems few got right: > the > elimination of the cross compilation for the packages. I was playing > with LFS for about a decade now from time to time and was also playing > with various other build systems: Gentoo, OpenWRT and recently > Openembedded/Yocto. I was quite excited about OE until I decided to > try > to port some packages to it. After some frustrations, I composed a > proper web search query and landed on Aborigianl Linux website :) Thanks. (It's hard to describe what the project does in a way people can find it without already knowing what they're looking for.) > I have a general question about the philosophy of AL: coming from LFS, > it seems that what AL is doing before launching QEMU is building a > native "host-independent" toolchain (in LFS terminology). That's one of the stages, yes. We'd happily use other people's toolchains (it's designed to be modular), but nobody else provides prebuilt binary _native_ toolchains, and we need those too. If our setup doesn't support a microblaze cross compiler, it probably won't support a microblaze native compiler, so using somebody else's cross compiler doesn't buy us much... :( > LFS builds the > toolchain under /tools, which means it is easy to discard it > afterwards. > Why doesn't AL follow the following procedure? Do you mean /tools on the host system? Because writing to that would require root access, and nothing that runs on the host requires root access. Do you mean in the system image? I did that for a while (commit 782 changed "NATIVE_TOOLS" directory to "ROOT_NODIRS"the last vestige of it was removed by commit http://landley.net/hg/aboriginal/rev/1411 , see also 1272, 783, and so on...) but it was confusing to explain and tended to be brittle. When the default rootfilesystem became squashfs (meaning you can run multiple emulator instances from the same root filesystem at once, don't have to worry about fsck, don't have to figure out how much extra writeable space to preallocate...), creating a new image involved populating a chroot directory under /home (which could be started with a snapshot of the current one, see the lfs-bootstrap.hdc image for an example, either at http://landley.net/hg/control-images or http;//landley.net/aboriginal/control-images/downloads/binaries/lfs-bootstrap.hdc) was the way to go, so having stuff in /tools on the original filesystem didn't help anymore. > Essentially what I want > is to build my own C library under nat ive buil d and link everything > against it, instead of linking against uClibc that is shipped by AL. I > am yet unsure on how easy this would be, provided that C library is > under /lib and not under /tools/lib. I'm trying to convert aboriginal over to use musl-libc Natively, if you want to use a different libc the most obvious approach is to build a chroot that uses it. Start by installing your new toolchain in a directory under /home in the system iamge (which dev-environment.sh attaches a 2 gigabyte hdb.img ext3 image to by default, so it's writeable and potentially persistent), then use that toolchain to build more stuff. It's still fiddly system building, but at least it's _native_ fiddly system building. I.E. you can follow linux from scratch mostly at that point, if you want. (modulo needing to use /home/tools instead of /tools because the root filesystem is squashfs and thus not writeable. If you want a writeable ext2 root filesystem you can set SYSIMAGE_TYPE=ext2 before running build.sh (either by modifying "config" or exporting it as an enviornment variable) and it'll produce one, and you'll probably want to set SYSIMAGE_HDA_MEGS as well. Or just look at the lfs-bootstrap.hdc image to use as an example, the native-build.sh script mounts it on /mnt and runs /mnt/init instead of providing a shell prompt. (Hit space when it prompts you to get a shell prompt anyway, that's an easy way to poke around in it. That's how I debug the native builds...) Rob 1382974495.0