From alessioigorbogani at gmail.com Fri Oct 5 14:47:26 2012 From: alessioigorbogani at gmail.com (Alessio Igor Bogani) Date: Fri, 05 Oct 2012 16:47:26 +0200 Subject: [Aboriginal] Build from version control system repos instead of from tarballs In-Reply-To: <504BF725.4010100@landley.net> References: <504BF725.4010100@landley.net> Message-ID: <506EF2FE.1020705@gmail.com> Hi Rob, On 09/09/2012 03:55, Rob Landley wrote: > On 09/04/2012 03:49 AM, Alessio Igor Bogani wrote: >> Hi All, >> >> Can Aboriginal grabs sources from git(Linux, musl, uclibc, busybox) >> and hg(toybox) repos instead of from tarballs? >> Thanks! [...] > I'm thinking about ways to simplify the package cache dance I mentioned > earlier. Possibly the package cache should be packages/cache and then > the procedure should be: > > 1) comment out the tarball URL in download.sh > 2) remove the patches you don't want applied from sources/patches > 3) git clone packages/cache/thingy Unfortunately it doesn't work. I have tried with the linux kernel so I commented out download.sh lines in this way: #URL=http://www.kernel.org/pub/linux/kernel/v3.x/linux-3.5.tar.bz2 \ #SHA1=d80e85147417605e125fc3fb263af85d7ffd1ad5 \ #ALT=http://kernel.org/pub/linux/kernel/v3.0/testing/linux-3.0-rc1.tar.bz2 \ #maybe_fork "download || dienow" Left sources/patches untouched. Cloned linux git tree into packages/cache/linux ./build.sh $ARCH results in: make[1]: Leaving directory `/home/alessio/Downloads/aboriginal-git-1.2.0/build/temp-i686-virtio/build-gcc/gcc' cleanup build-gcc cleanup gcc-core === ccwrap (i686-virtio simple-cross-compiler) === linux-headers (i686-virtio simple-cross-compiler) === linux (i686-virtio simple-cross-compiler) No tarball for linux Exiting due to errors (i686-virtio simple-cross-compiler linux) real 2m30.463s user 0m48.571s sys 0m19.117s Do you have any suggestions? Ciao, Alessio 1349448446.0 From rob at landley.net Fri Oct 5 15:39:00 2012 From: rob at landley.net (Rob Landley) Date: Fri, 5 Oct 2012 10:39:00 -0500 Subject: [Aboriginal] Build from version control system repos instead of from tarballs In-Reply-To: <506EF2FE.1020705@gmail.com> References: <504BF725.4010100@landley.net> <506EF2FE.1020705@gmail.com> Message-ID: Let's try that again actually cc:ing the list. (Not a fan of the gmail web interface. Need to fix my email...) On Fri, Oct 5, 2012 at 9:47 AM, Alessio Igor Bogani wrote: > Hi Rob, > > > On 09/09/2012 03:55, Rob Landley wrote: >> >> On 09/04/2012 03:49 AM, Alessio Igor Bogani wrote: >>> >>> Hi All, >>> >>> Can Aboriginal grabs sources from git(Linux, musl, uclibc, busybox) >>> and hg(toybox) repos instead of from tarballs? >>> Thanks! > > [...] > >> I'm thinking about ways to simplify the package cache dance I mentioned >> earlier. Possibly the package cache should be packages/cache and then >> the procedure should be: >> >> 1) comment out the tarball URL in download.sh >> 2) remove the patches you don't want applied from sources/patches >> 3) git clone packages/cache/thingy > > > Unfortunately it doesn't work. > > I have tried with the linux kernel so I commented out download.sh lines in > this way: > #URL=http://www.kernel.org/pub/linux/kernel/v3.x/linux-3.5.tar.bz2 \ > #SHA1=d80e85147417605e125fc3fb263af85d7ffd1ad5 \ > #ALT=http://kernel.org/pub/linux/kernel/v3.0/testing/linux-3.0-rc1.tar.bz2 \ > #maybe_fork "download || dienow" > > Left sources/patches untouched. Ah, sorry. I need to properly document this. If you comment out the URL= and maybe_fork "download || dienow" bits, it won't update the timestamp on the tarball in the packages directory, and then cleanup_oldfiles will delete all files in there that predate this run of download.sh (thinking they're old leftover versions that are no longer used.) (This only matters if you're putting an alt-tarball in the packages directory, which it doesn't sound like you're doing.) > Cloned linux git tree into packages/cache/linux You mean build/packages/linux ? I mentioned I was thinking about moving the package cache under the "packages" directory rather than the "build" directory, but neither is quite a good fit. The build directory is all the generated files, so you can "rm -rf build" and then re-run the build and that does a clean build. The "packages" directory is all the downloaded files that came from somewhere other than this source repository. The download.sh script populates that. Whether the extracted and patched source is a better fit for "build" (generated files) or "packages" (downloaded files) is an open question. > Do you have any suggestions? mkdir -p build/packages git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux build/packages/linux Try that? > Ciao, > Alessio 1349451540.0 From maillist-aboriginal at barfooze.de Sun Oct 7 11:10:21 2012 From: maillist-aboriginal at barfooze.de (John Spencer) Date: Sun, 07 Oct 2012 13:10:21 +0200 Subject: [Aboriginal] Dealing with distcc Message-ID: <5071631D.30406@barfooze.de> Rob, i'm playing around with distcc support in aboriginal, but there doesnt seem to be any compiler activity on the host system, and compilation is creeping incredibly slowly. only the qemu process itself is constantly using 100% cpu. i've put the equivalent cross-compiler-xxx/bin dir in $PATH, then started ./dev-env.sh. this script seems to set "CPUS" automatically to (physical cpus * 2). why *2 ? then there's another variable that could have an effect (you mentioned it on IRC once): KERNEL_EXTRA. what is that for? when qemu boots, it reports that distcc acceleration is enabled. can this message be trusted ? isn't some additional service required on the host (e.g. starting a distcc service) ? is there a way to debug if distcc on the host is called indeed ? now to the qemu system: which flags should be used when building stuff ? should i use make -j$NUMBER_OF_HOST_CPUS, or even CPUS*2, or make -j1 ? please shed some light. thanks. 1349608221.0 From maillist-aboriginal at barfooze.de Sun Oct 7 18:45:29 2012 From: maillist-aboriginal at barfooze.de (John Spencer) Date: Sun, 07 Oct 2012 20:45:29 +0200 Subject: [Aboriginal] Dealing with distcc In-Reply-To: <5071631D.30406@barfooze.de> References: <5071631D.30406@barfooze.de> Message-ID: <5071CDC9.5030202@barfooze.de> On 10/07/2012 01:10 PM, John Spencer wrote: > only the qemu process itself is constantly using 100% cpu. i guess i should use your toolchain and not my own to get distcc support XD i have now limited success using distcc: when i compile a bigger piece of sw i get "(distcc)" output in the beginning, but in the later phase it's only "gcc -foo bar.c" kinda stuff... btw: this line in ./dev-env... makes the autodetection mechanisms further down void (if you don't specify CPUS in your env, you'll always get 1): [ -z "$CPUS" ] && CPUS=1 1349635529.0 From maillist-aboriginal at barfooze.de Sun Oct 7 18:56:21 2012 From: maillist-aboriginal at barfooze.de (John Spencer) Date: Sun, 07 Oct 2012 20:56:21 +0200 Subject: [Aboriginal] Dealing with distcc In-Reply-To: <5071CDC9.5030202@barfooze.de> References: <5071631D.30406@barfooze.de> <5071CDC9.5030202@barfooze.de> Message-ID: <5071D055.60204@barfooze.de> On 10/07/2012 08:45 PM, John Spencer wrote: > i have now limited success using distcc: when i compile a bigger piece > of sw i get "(distcc)" output in the beginning, but in the later phase > it's only "gcc -foo bar.c" kinda stuff... well, actually it's distcc[203] (dcc_build_somewhere) Warning: failed to distribute, running locally instead distcc[202] (dcc_build_somewhere) Warning: failed to distribute, running locally instead distcc[204] (dcc_build_somewhere) Warning: failed to distribute, running locally instead is it possible that all aboriginal instances use the same distcc port ? i currently have armv4tl, armv6 and mips running... 1349636181.0 From rob at landley.net Sun Oct 7 23:29:03 2012 From: rob at landley.net (Rob Landley) Date: Sun, 7 Oct 2012 18:29:03 -0500 Subject: [Aboriginal] Dealing with distcc In-Reply-To: <5071D055.60204@barfooze.de> References: <5071631D.30406@barfooze.de> <5071CDC9.5030202@barfooze.de> <5071D055.60204@barfooze.de> Message-ID: On Sun, Oct 7, 2012 at 1:56 PM, John Spencer wrote: > On 10/07/2012 08:45 PM, John Spencer wrote: >> >> i have now limited success using distcc: when i compile a bigger piece >> of sw i get "(distcc)" output in the beginning, but in the later phase >> it's only "gcc -foo bar.c" kinda stuff... > > > well, actually it's > distcc[203] (dcc_build_somewhere) Warning: failed to distribute, running > locally instead > distcc[202] (dcc_build_somewhere) Warning: failed to distribute, running > locally instead > distcc[204] (dcc_build_somewhere) Warning: failed to distribute, running > locally instead So the QEMU instance of the distcc c wrapper is trying to send it to host, but the host distccd isn't responding, so it retries locally. > is it possible that all aboriginal instances use the same distcc port ? > i currently have armv4tl, armv6 and mips running... Simultaneously, as the first thing you're trying? Try to get one target to distribute distcc, and only add a second if that works. In theory there's code that sets each instance up on a separate port based on PID (in dev-environment.sh). In practice, distcc is notoriously brittle and provides _horrible_ feedback about what it's doing. One of my todo items is teaching my ccwrap code (toys/ccwrap.c) to do distcc style distribution itself. Other problems with distcc are that if you give it a combined command line (ala "gcc hello.c -o hello $LINKOPTS" it won't split it into "distcc gcc -c -o tempnam.o; gcc tempnam.o -o hello $LINKOPTS" and thus the compile of something like toybox (gcc file1.c file2.c file3.c... -o toybox") happens entirely on the target, with no benefit from distcc at all. I have many, many things I would like to do. I put in my two weeks notice at work last monday because for the past year, my job has prevented me from doing _any_ of them. I need to find a new job that lets me put at least some time into this stuff. (Although toybox gets priority until it hits 1.0...) Rob 1349652543.0 From rob at landley.net Mon Oct 8 05:49:47 2012 From: rob at landley.net (Rob Landley) Date: Mon, 8 Oct 2012 00:49:47 -0500 Subject: [Aboriginal] I aten't dead. Message-ID: You may have noticed that my day job's been leaving me with zero time and energy to do open source work, and I recently decided it wasn't what I wanted to do with my life and gave my two weeks notice, so I should have some time the second half of the month to catch up on stuff before I have to go look for a contract. My priorities for Aboriginal are getting toybox and musl integrated, and weaning the code off of busybox and uClibc. After that probably getting LFS updated to current, and then I need to tackle the toolchain issue. Anybody else want to throw a suggestion into the ring? Rob 1349675387.0 From onefang at gmail.com Mon Oct 8 06:17:18 2012 From: onefang at gmail.com (David Seikel) Date: Mon, 8 Oct 2012 16:17:18 +1000 Subject: [Aboriginal] I aten't dead. In-Reply-To: References: Message-ID: <20121008161718.1cf2de9f.onefang@gmail.com> On Mon, 8 Oct 2012 00:49:47 -0500 Rob Landley wrote: > My priorities for Aboriginal are getting toybox and musl integrated, > and weaning the code off of busybox and uClibc. Seems like a good idea to me. -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From maillist-aboriginal at barfooze.de Mon Oct 8 16:48:09 2012 From: maillist-aboriginal at barfooze.de (John Spencer) Date: Mon, 08 Oct 2012 18:48:09 +0200 Subject: [Aboriginal] Dealing with distcc In-Reply-To: References: <5071631D.30406@barfooze.de> <5071CDC9.5030202@barfooze.de> <5071D055.60204@barfooze.de> Message-ID: <507303C9.2030702@barfooze.de> On 10/08/2012 01:29 AM, Rob Landley wrote: > On Sun, Oct 7, 2012 at 1:56 PM, John Spencer >> distcc[204] (dcc_build_somewhere) Warning: failed to distribute, running >> locally instead > > So the QEMU instance of the distcc c wrapper is trying to send it to > host, but the host distccd isn't responding, so it retries locally. there seems to be something horrible broken with the qemu-system-mips image i'm using (iirc aboriginal 1.1). later (while building stuff) i got this here: 2012.10.07 19:33:11 building stage0_gcc3 (/home/sabotage.bld/src/build/build_stage0_gcc3.sh) -> /home/sabotage.bld/src/logs/build_stage0_gcc3.log ------------[ cut here ]------------ WARNING: at net/sched/sch_generic.c:255 dev_watchdog+0x170/0x280() NETDEV WATCHDOG: eth0 (pcnet32): transmit queue 0 timed out Call Trace: [<80106a68>] dump_stack+0x8/0x34 [<8011f324>] warn_slowpath_common+0x70/0x9c [<8011f3d8>] warn_slowpath_fmt+0x2c/0x38 [<802dde54>] dev_watchdog+0x170/0x280 [<8012b068>] run_timer_softirq+0x148/0x1f0 [<80124fb8>] __do_softirq+0xa0/0x140 [<801251a0>] do_softirq+0x48/0x68 [<80125200>] irq_exit+0x40/0x90 [<8010178c>] ret_from_irq+0x0/0x4 ---[ end trace 2cb711632f922f00 ]--- pcnet32: eth0: transmit timed out, status 07fb, resetting pcnet32: eth0: transmit timed out, status 07fb, resetting pcnet32: eth0: transmit timed out, status 07fb, resetting [etc] i had to disable distcc (no crosscompiler in path, CPUS=1) entirely to make it work robustly (no other network stuff used). interestingly this is also the single system which died due to OOM while building gcc 4.5.3. i had to add swap as HDC (making a swapfile inside qemu failed due to EINVAL delivered to swapon command). i didn't experience any of these issues using the arm system images. > >> is it possible that all aboriginal instances use the same distcc port ? >> i currently have armv4tl, armv6 and mips running... > > Simultaneously, as the first thing you're trying? Try to get one > target to distribute distcc, and only add a second if that works. > yeah it seems to work for the arm system images. it looks like the mips systemimage has a borked network stack. eventually i'll try your currently mips image later this week, maybe that fixes it. P.S. have you seen the bug i reported about CPUS always getting set to 1 unless you pass it manually ? P.P.S. have you archived older versions of prebuilt system images and cross compilers somewhere ? P.P.P.S. i'd be glad if you pushed an aboriginal release with the "new" binutils 2.17.xxx installed soon. the lack of -Bsymbolic-functions in the old version makes it impossible to use dynamic musl, and can lead to subtle bugs even in some statically linked apps. --JS 1349714889.0 From maillist-aboriginal at barfooze.de Wed Oct 10 00:04:07 2012 From: maillist-aboriginal at barfooze.de (John Spencer) Date: Wed, 10 Oct 2012 02:04:07 +0200 Subject: [Aboriginal] missing applets in current 1.20 system images Message-ID: <5074BB77.20303@barfooze.de> (mipsel:3) /home # free -m /bin/ash: free: not found (mipsel:3) /home # /bin/busybox mkswap mkswap: applet not found (mipsel:3) /home # swapon /bin/ash: swapon: not found 1349827447.0 From maillist-aboriginal at barfooze.de Wed Oct 10 01:01:51 2012 From: maillist-aboriginal at barfooze.de (John Spencer) Date: Wed, 10 Oct 2012 03:01:51 +0200 Subject: [Aboriginal] missing applets in current 1.20 system images In-Reply-To: <5074BB77.20303@barfooze.de> References: <5074BB77.20303@barfooze.de> Message-ID: <5074C8FF.5050908@barfooze.de> On 10/10/2012 02:04 AM, John Spencer wrote: > (mipsel:3) /home # free -m > /bin/ash: free: not found > (mipsel:3) /home # /bin/busybox mkswap > mkswap: applet not found > (mipsel:3) /home # swapon > /bin/ash: swapon: not found sha*sum is missing as well... and surely others i missed. what happened here ? a failed attempt of toybox takeover ? a system without these tools is not of much use. where can i download the working 1.1 system images and corresponding cross-compilers please ? [ 3rd time i ask this now ] 1349830911.0 From rob at landley.net Thu Oct 25 06:48:57 2012 From: rob at landley.net (Rob Landley) Date: Thu, 25 Oct 2012 01:48:57 -0500 Subject: [Aboriginal] I have email again! Sort of. Message-ID: <1351147737.16660.0@driftwood> Ok, I quit the job at polycom (which was eating my life to the exclusion of all else, last day was the 14th), and finally repaired my email (the saga of which was unreasonably long and elaborate). I'm not exactly _happy_ with the new email client but it sort of works if you squint, which is an improvement over what I had. Now I have at least a month of backlog to catch up on. Sorry for the radio silence. Rob 1351147737.0 From rob at landley.net Fri Oct 26 01:27:08 2012 From: rob at landley.net (Rob Landley) Date: Thu, 25 Oct 2012 20:27:08 -0500 Subject: [Aboriginal] missing applets in current 1.20 system images In-Reply-To: <5074BB77.20303@barfooze.de> (from maillist-aboriginal@barfooze.de on Tue Oct 9 19:04:07 2012) Message-ID: <1351214828.16660.8@driftwood> I might already have replied to this one, but since gmail eats copies of messages lists send back to me (because google has strong opinions about what I should and shouldn't see, which are often empirically wrong), and I'm not online to check the web archive, I dunno. On 10/09/2012 07:04:07 PM, John Spencer wrote: > (mipsel:3) /home # free -m > /bin/ash: free: not found > (mipsel:3) /home # /bin/busybox mkswap > mkswap: applet not found > (mipsel:3) /home # swapon > /bin/ash: swapon: not found Yup. There's a reason for that. I'm migrating the code from busybox to toybox. (Also from uClibc to musl, but I don't have to write that one.) In preparation for this, I stripped down busybox from "defconfig" to sources/baseconfig-busybox with just the list of commands the build actually uses (as determiend by more/record-commands.sh). This includes the set of commands used to boot the target system, and to build Linux From Scratch under the result. I'm then gradually replacing those busybox commands with toybox versions. Since then, I've been adding back commands like "top" that the build doesn't use, but which I really miss if they're not there. You've just spotted three more (which are already in the repository version of toybox, but not in the older release version aboriginal's using). If you want the old "kitchen sink" configuration of busybox, it's: BUSYBOX=1 ./build.sh i686 That'll use a defconfig version of busybox instead of the stripped down version plus toybox. Rob 1351214828.0 From rob at landley.net Fri Oct 26 01:44:48 2012 From: rob at landley.net (Rob Landley) Date: Thu, 25 Oct 2012 20:44:48 -0500 Subject: [Aboriginal] Dealing with distcc In-Reply-To: <5071D055.60204@barfooze.de> (from maillist-aboriginal@barfooze.de on Sun Oct 7 13:56:21 2012) Message-ID: <1351215888.16660.9@driftwood> On 10/07/2012 01:56:21 PM, John Spencer wrote: > On 10/07/2012 08:45 PM, John Spencer wrote: > > i have now limited success using distcc: when i compile a bigger > piece > > of sw i get "(distcc)" output in the beginning, but in the later > phase > > it's only "gcc -foo bar.c" kinda stuff... > > well, actually it's > distcc[203] (dcc_build_somewhere) Warning: failed to distribute, > running > locally instead > distcc[202] (dcc_build_somewhere) Warning: failed to distribute, > running > locally instead > distcc[204] (dcc_build_somewhere) Warning: failed to distribute, > running > locally instead > > is it possible that all aboriginal instances use the same distcc port > ? > i currently have armv4tl, armv6 and mips running... It shouldn't. It should randomize based on the PID, within a port range that's otherwise unused. The logic for all this is in sources/ toys/unique-port.sh (which gets sucked into dev-environment.sh in the system images). The first 8192 ports are restricted to root only by the kernel (it used to be 1024 but the kernel's gotten increasingly conservative in its old age). In addition, a range of ports are dynamically assigned to normal connections. So what the code does is it tries to use 8192 through the start of the dynamic range. If there are fewer than 8 entries in that, it tries to use the end of the range through 65535. If there are fewer than 8 entries in _that_, it punts and goes "ok, 16384 through 32768, hope we don't collide with anything". It then takes the current PID, modulo the range, plus the start of the range, and uses that as the port number. This is not guaranteed to work, but it is at least _trying_ to find an unused port. (The problem with dynamically assigning a port is we don't get to see what the port number _is_ because we backgrounded the daemon. Last I checked dhcpd doesn't have an inetd mode so we can't handle this more gracefully, or at least I haven't figured out how yet.) Rob 1351215888.0 From onefang at gmail.com Fri Oct 26 05:14:20 2012 From: onefang at gmail.com (David Seikel) Date: Fri, 26 Oct 2012 15:14:20 +1000 Subject: [Aboriginal] Getting around gmail "features". Was: missing applets in current 1.20 system images In-Reply-To: <1351214828.16660.8@driftwood> References: <5074BB77.20303@barfooze.de> <1351214828.16660.8@driftwood> Message-ID: <20121026151420.2f130950.onefang@gmail.com> On Thu, 25 Oct 2012 20:27:08 -0500 Rob Landley wrote: > I might already have replied to this one, but since gmail eats copies > of messages lists send back to me (because google has strong opinions > about what I should and shouldn't see, which are often empirically > wrong), and I'm not online to check the web archive, I dunno. That's why I subscribe a read only and a separate write only account to email lists. gmail filters the posts the write only account makes, but that's OK, I don't use that account to read the lists. The read only account never posts, so it sees the entire list and deals with it fine. People replying or CCing replies to the write only account, those go through the normal email process and bypass the list anyway. So I get to see them to. Plus I have a filter that removes duplicates for those that reply to the list AND CC me, I guess based on the theory that people never the read lists they post to. shrugs Back in the day when gmail was beta and invite only, I (and others) complained long and loud that this was a bug, but google insists that it's a feature. Using read and write accounts is STILL the only way around that. An option to turn that off would make everyone happy. I guess that's just too hard for google to implement. Seriously considering just running my own private email server. -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From rob at landley.net Sat Oct 27 05:58:51 2012 From: rob at landley.net (Rob Landley) Date: Sat, 27 Oct 2012 00:58:51 -0500 Subject: [Aboriginal] Getting around gmail "features". Was: missing applets in current 1.20 system images In-Reply-To: <20121026151420.2f130950.onefang@gmail.com> (from onefang@gmail.com on Fri Oct 26 00:14:20 2012) References: <5074BB77.20303@barfooze.de> <1351214828.16660.8@driftwood> <20121026151420.2f130950.onefang@gmail.com> Message-ID: <1351317531.16093.2@driftwood> On 10/26/2012 12:14:20 AM, David Seikel wrote: > Back in the day when gmail was beta and invite only, I (and others) > complained long and loud that this was a bug, but google insists that > it's a feature. Using read and write accounts is STILL the only way > around that. An option to turn that off would make everyone happy. > I > guess that's just too hard for google to implement. > > Seriously considering just running my own private email server. I used to run an email server, but it was too much like work. I'm halfway to writing my own email _client_ and don't have time to do that. (It's a failure mode I try to avoid. I do NOT have time.) Rob 1351317531.0 From rob at landley.net Sat Oct 27 06:51:52 2012 From: rob at landley.net (Rob Landley) Date: Sat, 27 Oct 2012 01:51:52 -0500 Subject: [Aboriginal] [musl] microblaze port committed In-Reply-To: <20120930000141.0167901f.idunham@lavabit.com> (from idunham@lavabit.com on Sun Sep 30 02:01:41 2012) Message-ID: <1351320712.16093.6@driftwood> My email was broken forever, but I've built a new one out of duck tape and am attempting to deal with the backlog: On 09/30/2012 02:01:41 AM, Isaac Dunham wrote: > On Sat, 29 Sep 2012 01:41:57 -0400 > Rich Felker wrote: > > > Hi all, > > > > I've committed the initial (and seemingly fully-working) version of > > the microblaze port. Several caveats: > > > > 1. Upstream binutils has a serious bug in gas whereby the > relocations > > it generates for symbols with local weak definitions cannot be > > resolved by the linker. Patch at: > > http://permalink.gmane.org/gmane.linux.uclinux.microblaze/11155 My current understanding is this bug was introduced _after_ 2.17, so doesn't hit my frozen toolchain? > > 2. The toolchain binaries from Xilinx seem to produce .o files > > incompatible with standard binutils, and vice versa. Pick one or > > the other; don't mix them. > For Landley's sake, I'll mention what we discussed on IRC (+ some > details): > -The Xilinx toolchain is binutils 2.16 + gcc 4.1.2 (GPLv2); binutils > support was merged upstream in 2009, so Aboriginal would need the > Xilinx EDK toolchain (see http://git.xilinx.com/? > p=mb_gnu.git;a=summary > ofor source) I've had a xylinx toolchain of approximately that vintage in my todo heap for a couple years, the problem is their changes broke arm. (If you build an arm toolchain from those sources, it didn't work.) I can try refreshing from xylinx git, but getting basic musl support in is comes first. (Maybe 1/3 through ccwrap.c rewrite, I'm doing the new one to be a reasonable base for later qcc and distcc replacement work.) > -Rich says that on microblaze, due to the way relocations are > handled, > using -Bsymbolic instead of -Bsymbolic-functions gives a libc.so that > is either minimally broken or fully operational. > (Mips also works with -Bsymbolic.) I already updated binutils to the new version musl needs. I can build musl, it's just my ccwrap is full of uClibc baked-in assumptions that need to go away, and the code's brittle enough I'm just writing a fresh one. > -The possibility of using mapfiles instead of -Bsymbolic-functions > was > mentioned, but Rich doubts that they were supported in older > binutils. > > > 3. Threads are untested because qemu is broken, at least my version > of > > qemu (app-level emu) has a bug where clone() does not advance > the > > program counter properly in the child, so it starts a > forkbomb-like > > cascade of clone syscalls (thankfully just linear growth rather > > than exponential). > Ow. > > Rob, any thoughts on an Aboriginal port? I want to do one, but I want to start with x86, then arm, then the other existing platforms I can compare against a known working version of. Rob 1351320712.0 From rob at landley.net Sat Oct 27 19:09:10 2012 From: rob at landley.net (Rob Landley) Date: Sat, 27 Oct 2012 14:09:10 -0500 Subject: [Aboriginal] binutils tarballs... In-Reply-To: <18012.132.241.18.69.1348713122.squirrel@lavabit.com> (from idunham@lavabit.com on Wed Sep 26 21:32:02 2012) References: <18012.132.241.18.69.1348713122.squirrel@lavabit.com> Message-ID: <1351364950.22665.3@driftwood> On 09/26/2012 09:32:02 PM, idunham at lavabit.com wrote: > Hello, > Binutils release-style tarballs can _almost_ be built using > > make -f src-release binutils.tar.bz2 MAKEINFO=makeinfo #or true > > There will be a binutils-2.17.50.tar.bz2 file, but I forget whether it > lands in $PWD or ../ > > However, there is still one catch: > one file (something/*.texinfo, IIRC) isn't built. > You'd need to rename it or manually invoke makeinfo, then make again. Is the binutils tarball aboriginal-hg is currently using working for you? > Also, you may want to check that all the needed files are included > (diff/sort -u the list of files in 2.17 & 2.17a for a general idea). > This > is allegedly the reason that the original tarballs got removed. Nah, the extra files they added (which are under gplv3) were never needed to build anything. (That's why binutils 2.17 could ship without them). What they added were the "preferred form for editing". A bit like shipping the makefiles without shipping the autotools files. Since I prefer hacking the makefiles _anyway_, I'm pretty happy ignoring a source format I never needed, never wanted, and don't care about. "Preferred" is a judgement call here, if I need to hack up something some other tool generated, I'm pretty ok with this. They shipped it for years without _noticing_, and then couldn't track down versions under the original license, so it can't be all _that_ preferred now can it? Rob 1351364950.0 From rob at landley.net Tue Oct 30 02:36:17 2012 From: rob at landley.net (Rob Landley) Date: Mon, 29 Oct 2012 21:36:17 -0500 Subject: [Aboriginal] Build from version control system repos instead of from tarballs In-Reply-To: (from alessioigorbogani@gmail.com on Tue Sep 4 03:49:07 2012) Message-ID: <1351564577.9254.1@driftwood> On 09/04/2012 03:49:07 AM, Alessio Igor Bogani wrote: > Hi All, > > Can Aboriginal grabs sources from git(Linux, musl, uclibc, busybox) > and hg(toybox) repos instead of from tarballs? > Thanks! I think I already replied to this? Talking about using the alt- infrastructure and populating the package cache from the git repo, and how the tricky bit at the design level is applying patches to the repo? (Really you need a source control branch with the appropriate patches...) I have a todo item to figure out a clean way to do this. Hmmm... Really what I want to do is say "URL=git://busybox.net/busybox" in download.sh, have that clone the relevant repo under packages (so packages/busybox would be a directory), and then just use that as the package cache version. The question is, when do I re-pull? (Every time you run download.sh?) Rob 1351564577.0 From rob at landley.net Tue Oct 30 02:42:01 2012 From: rob at landley.net (Rob Landley) Date: Mon, 29 Oct 2012 21:42:01 -0500 Subject: [Aboriginal] gfortran In-Reply-To: (from brian.fix@gmail.com on Thu Aug 30 21:08:03 2012) Message-ID: <1351564921.9254.2@driftwood> On 08/30/2012 09:08:03 PM, Brian Fix wrote: > Hi Aboriginal Users, I think I replied to this before my netbook was stolen, but in case I didn't... > I found aboriginal looking into cross compiling python for my > android. I've > seen multiple threads and projects on how to do it, they usually > involve a > messy patch to apply to a specific version, and multiple stages of > configuring and building with very long argument lists. > > Not the case with aboriginal, a simple: > > ./configure --without-threads --prefix=/home/python_blah > make > make install > > copied the results, and the dynamic libraries, onto my android > device, ran > elfpatch on the python to point the python to the new library > locations, > and i was done. > > Now I'm spoiled, I want numpy, scipy and matplotlib this easily! > > Unfortunately, numpy/scipy have dependencies which require a Fortran > compiler, so here's my problem. If I did reply, I probably pointed you at the control-images stuff? http://landley.net/aboriginal/control-images Specifically the linux from scratch control image. (Which I really need to update to the new version...) > I've downloaded the complete gcc tarball with gfortran and all other > languages from their website and tried the same simple approach: > > tar xvzf gcc-4.6.0.tar.gz > cd gcc-4.6.0/ > ./contrib/download_prerequisits > mkdir gcc_build > cd gcc_build > ../gcc-a.b.c/configure > make > > but I get a build error in libgomp: > > checking whether make sets $(MAKE)... yes > checking for armv6l-unknown-linux-gnueabi-gcc... > /home/gcc_build/./gcc/xgcc > -B/home/gcc_build/./gcc/ > -B/usr/local/armv6l-unknown-linux-gnueabi/bin/ > -B/usr/local/armv6l-unknown-linux-gnueabi/lib/ -isystem > /usr/local/armv6l-unknown-linux-gnueabi/include -isystem > /usr/local/armv6l-unknown-linux-gnueabi/sys-include > checking for C compiler default output file name... > configure: error: in > `/home/gcc_build/armv6l-unknown-linux-gnueabi/libgomp': > configure: error: C compiler cannot create executables > See `config.log' for more details. > make[2]: *** [configure-stage1-target-libgomp] Error 77 > make[2]: Leaving directory `/home/gcc_build' > make[1]: *** [stage1-bubble] Error 2 > make[1]: Leaving directory `/home/gcc_build' > make: *** [all] Error 2 > (armv6l:1) /home/gcc_build # What did config.log say? > My question is multipart: > > should I be using gcc-4.6.0 or some other version more agreeable with > aboriginal? > > Is there a better way to do this? > > I notices a series of gcc patches in the aboriginal-1.2.0.tar.gz, > should I > download 4.2.1 source apply these patches and compile? You should be able to build another gcc under the target system. I'm reasonably sure I've done it before. (There's a script to do so in the lfs build, but it and the glibc one aren't enabled at the moment.) I'm in the middle of switching out uClibc for musl on all the targets, which should make building glibc (and thus a new toolchain) natively on target much easier. Alas, it means that for the moment my local copies have parts on the floor and dangling wires...; Rob 1351564921.0 From maillist-aboriginal at barfooze.de Tue Oct 30 02:55:08 2012 From: maillist-aboriginal at barfooze.de (John Spencer) Date: Tue, 30 Oct 2012 03:55:08 +0100 Subject: [Aboriginal] ext 4 errors using mipsel system image Message-ID: <508F418C.7030202@barfooze.de> FYI: i just got a couple of these when removing an object file from /home and rebuilding it manually. EXT4-fs error (device hdb): ext4_mb_generate_buddy:741: group 921, 3846 clusters in bitmap, 4417 in gd 1351565708.0 From onefang at gmail.com Tue Oct 30 08:32:59 2012 From: onefang at gmail.com (David Seikel) Date: Tue, 30 Oct 2012 18:32:59 +1000 Subject: [Aboriginal] Build from version control system repos instead of from tarballs In-Reply-To: <1351564577.9254.1@driftwood> References: <1351564577.9254.1@driftwood> Message-ID: <20121030183259.2ce9dfd7.onefang@gmail.com> On Mon, 29 Oct 2012 21:36:17 -0500 Rob Landley wrote: > On 09/04/2012 03:49:07 AM, Alessio Igor Bogani wrote: > > Hi All, > > > > Can Aboriginal grabs sources from git(Linux, musl, uclibc, busybox) > > and hg(toybox) repos instead of from tarballs? > > Thanks! > > I think I already replied to this? Talking about using the alt- > infrastructure and populating the package cache from the git repo, > and how the tricky bit at the design level is applying patches to the > repo? (Really you need a source control branch with the appropriate > patches...) > > I have a todo item to figure out a clean way to do this. > > Hmmm... > > Really what I want to do is say "URL=git://busybox.net/busybox" in > download.sh, have that clone the relevant repo under packages (so > packages/busybox would be a directory), and then just use that as > the package cache version. > > The question is, when do I re-pull? (Every time you run download.sh?) I guess there are two cases. You are pulling from a source repo, and you just want the latest head. Or you want a specific revision. I would suggest the addition of a "REV=foobar". In it's absence, just pull on every run, to get the latest head. With that variable, treat git like a tarball that can be cached, pull once, keep it cached for ever more. Under the "just pull the latest" scenario, the sha1 is meaningless, put under the "pull a specific revision" scenario, the sha1 should be for the entire directory pulled. Though in the case of git, the revision "number" IS part or all of an sha1 for the directory. So perhaps for git REV would equal SHA1 anyway, but not all source code repo software does that. So if you want something more generic, keep REV and SHA1 different beasts. Just my two cents before I actually wake up properly. -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From alessioigorbogani at gmail.com Tue Oct 30 08:49:56 2012 From: alessioigorbogani at gmail.com (Alessio Igor Bogani) Date: Tue, 30 Oct 2012 09:49:56 +0100 Subject: [Aboriginal] Build from version control system repos instead of from tarballs In-Reply-To: <1351564577.9254.1@driftwood> References: <1351564577.9254.1@driftwood> Message-ID: <508F94B4.8020508@gmail.com> Hi Rob, On 30/10/2012 03:36, Rob Landley wrote: > On 09/04/2012 03:49:07 AM, Alessio Igor Bogani wrote: [...] >> Can Aboriginal grabs sources from git(Linux, musl, uclibc, busybox) >> and hg(toybox) repos instead of from tarballs? [...] > populating the package cache from the git repo, and [...] > Really what I want to do is say "URL=git://busybox.net/busybox" in > download.sh, have that clone the relevant repo under packages (so > packages/busybox would be a directory), and then just use that as the > package cache version. > > The question is, when do I re-pull? (Every time you run download.sh?) Never anymore. I would want use a specific tag or commit (i.e. git reset --hard vX.Y) moreover I would want decide when update and switch to a different (updated) tag or commit. Personally I don't use tarballs at all anymore: I'll download the project's repo and build from it. I'm very happy if Aboriginal let me work in this way. For example (sorry in git-ish): git clone aboriginal cd aboriginal/ && ls build.sh config cross-compiler.sh .. (these should works with any version and doesn't care if gcc is 4.2.2 or 4.7.1) ls package/linux (<- this is a submodule) git submodule init git submodule update cd package/linux git reset --hard origin (reset to the Linus' HEAD) (don't change anything other aka let's use Rob's default versions) ./build i686-virtio Why I bother with this crazy request? Because In this way I can handle multi-level changes in a very simple way. So if I want add a feature which needs to change/patch kernel, libc and application I could treat this like a single atomic commit. Ciao, Alessio 1351586996.0 From rob at landley.net Tue Oct 30 22:22:25 2012 From: rob at landley.net (Rob Landley) Date: Tue, 30 Oct 2012 17:22:25 -0500 Subject: [Aboriginal] Build from version control system repos instead of from tarballs In-Reply-To: <508F94B4.8020508@gmail.com> (from alessioigorbogani@gmail.com on Tue Oct 30 03:49:56 2012) Message-ID: <1351635745.29012.4@driftwood> On 10/30/2012 03:49:56 AM, Alessio Igor Bogani wrote: > Hi Rob, > > On 30/10/2012 03:36, Rob Landley wrote: >> On 09/04/2012 03:49:07 AM, Alessio Igor Bogani wrote: > [...] >>> Can Aboriginal grabs sources from git(Linux, musl, uclibc, busybox) >>> and hg(toybox) repos instead of from tarballs? > [...] >> populating the package cache from the git repo, and > [...] >> Really what I want to do is say "URL=git://busybox.net/busybox" in >> download.sh, have that clone the relevant repo under packages (so >> packages/busybox would be a directory), and then just use that as the >> package cache version. >> >> The question is, when do I re-pull? (Every time you run >> download.sh?) > > Never anymore. > > I would want use a specific tag or commit (i.e. git reset --hard > vX.Y) moreover I would want decide when update and switch to a > different (updated) tag or commit. > > Personally I don't use tarballs at all anymore: I'll download the > project's repo and build from it. I'm very happy if Aboriginal let me > work in this way. It currently does, using a two step process: 1) Edit "configure" and add the packages you want to do git versions of to the USE_ALT variable. (USE_ALT=busybox,uClibc,linux) 2) Then check out your repos into build/packages/alt-busybox and build/packages/alt-uClibc and so on. What I was talking about was coming up with a more obvious way to do it, which doesn't involve quite so much knowledge of the guts of aboriginal's build to understand what's going on. (Basically the above manually populates the package cache. Using the alt- slot for the package disables the normal download and packaging behavior.) > For example (sorry in git-ish): > > git clone aboriginal I'm vaguely interested in having a git mirror of the mercurial repo, but I have no interest in hosting any of my projects in git. The git user interface is appalling, and their solution to all their problems is to expose implementation details to the user. > cd aboriginal/ && ls > build.sh config cross-compiler.sh .. (these should works with any > version and doesn't > care if gcc is 4.2.2 or 4.7.1) The build script required for 4.2 and 4.7 are _totally_different_. For one thing, newer gcc versions have something like 4 prerequisite packages you have to build first, requirements 4.2 didn't have. Unfortunately, your desire doesn't match reality there. > ls package/linux (<- this is a submodule) > git submodule init > git submodule update > cd package/linux I can add the ability to put extracted source in "packages/linux" and build from that, and I can probably teach download.sh to leave it alone if it's there. But the aboriginal build would be orthogonal to where that source came from. You can create your own git repo full of submodules for all the packages if you like, I'm not doing it. Right now, the only thing that cares where the contents of the "packages" directory came from is download.sh. The rest of the build doesn't. I intend to keep it orthogonal: git as an alternative to download.sh. > git reset --hard origin (reset to the Linus' HEAD) > (don't change anything other aka let's use Rob's default versions) > ./build i686-virtio How about this: I can modify the scripts so that if there's a directory under packages with the package name, we use that source verbatim. (No patches, no nothing.) Otherwise, we fall back to the current behavior. (Basically it's just implementing a search path for the package cache with the first entry searched before attempting to _populate_ the package cache.) That would mean if you "git checkout git://linux packages/linux" the build uses that git repo, and patching/updating it is your problem. Does that sound reasonable? I can provide an example script that would do a checkout of all the package versions we're using, and then the aboriginal build would stay out of your way. > Why I bother with this crazy request? Because In this way I can > handle multi-level > changes in a very simple way. So if I want add a feature which needs > to change/patch > kernel, libc and application I could treat this like a single atomic > commit. Which would never go upstream. My build uses clean vanilla release versions of upstream packages, and separates out all the patches (and I try to send those patches upstream so I don't have to carry them, with varying success). This is a design decision, so you can see what I'm using and what I did to it. Your suggestion makes things easier for you at the expense of making it harder for anybody else to see what you did or reproduce it outside your build system. That's fine for your purposes, but that's not what I'm after. I'm not _trying_ to scale the aboriginal build to compile more packages in a more complciated way. The point of the thing is to bootstrap the simplest possible native environment on target, and then let you build natively under that. I want to add more targets (hexagon, microblaze, get m68k working), and I want to expand the control-images repository to include more example packages you can build (especially automated distro bootstraps). But if you want a giant hairball that cross-compiles 300 packages without verifying that any of them actually runs, you want buildroot. I'm happy to make the build more flexible, and I _want_ my code to to stay out of your way so you can do what you want to do. I want to make it trivially easy to overlay "git submodule" on top of the build, but my build is not becoming dependent on git submodule. Rob 1351635745.0 From rob at landley.net Wed Oct 31 00:50:24 2012 From: rob at landley.net (Rob Landley) Date: Tue, 30 Oct 2012 19:50:24 -0500 Subject: [Aboriginal] ext 4 errors using mipsel system image In-Reply-To: <508F418C.7030202@barfooze.de> (from maillist-aboriginal@barfooze.de on Mon Oct 29 21:55:08 2012) Message-ID: <1351644624.29012.8@driftwood> On 10/29/2012 09:55:08 PM, John Spencer wrote: > FYI: i just got a couple of these when removing an object file from > /home > and rebuilding it manually. > > EXT4-fs error (device hdb): ext4_mb_generate_buddy:741: group 921, > 3846 clusters in bitmap, 4417 in gd Yeah, there's a kernel bug. They're hashing out the correct fix, last I saw was: http://lkml.indiana.edu/hypermail/linux/kernel/1210.3/02995.html (Basically ext4's been unstable since 3.6.3, people are just now noticing.) I have 3.6 ready to go in the repo, but would like a patch for this first... Rob 1351644624.0 From rob at landley.net Wed Oct 31 00:53:18 2012 From: rob at landley.net (Rob Landley) Date: Tue, 30 Oct 2012 19:53:18 -0500 Subject: [Aboriginal] Build from version control system repos instead of from tarballs In-Reply-To: <20121030183259.2ce9dfd7.onefang@gmail.com> (from onefang@gmail.com on Tue Oct 30 03:32:59 2012) References: <1351564577.9254.1@driftwood> <20121030183259.2ce9dfd7.onefang@gmail.com> Message-ID: <1351644798.29012.9@driftwood> On 10/30/2012 03:32:59 AM, David Seikel wrote: > On Mon, 29 Oct 2012 21:36:17 -0500 Rob Landley > wrote: > > The question is, when do I re-pull? (Every time you run > download.sh?) > > I guess there are two cases. You are pulling from a source repo, and > you just want the latest head. Or you want a specific revision. I > would suggest the addition of a "REV=foobar". In it's absence, just > pull on every run, to get the latest head. With that variable, treat > git like a tarball that can be cached, pull once, keep it cached for > ever more. > > Under the "just pull the latest" scenario, the sha1 is meaningless, > put > under the "pull a specific revision" scenario, the sha1 should be for > the entire directory pulled. Though in the case of git, the revision > "number" IS part or all of an sha1 for the directory. So perhaps for > git REV would equal SHA1 anyway, but not all source code repo software > does that. So if you want something more generic, keep REV and SHA1 > different beasts. > > Just my two cents before I actually wake up properly. I just did a "remove the alt- functionality" pass, which I'm testing, and next I'm adding logic so that if a directory packages/package exists for a package, it'll ignore the tarball and use that directly instead. (In this case it won't patch it, you'll have to patch it yourself.) This basically means you can supply your own extracted source for a package (git repo, mercurial, subversion: I don't care) and it'll just use it. I'm also taking out the "hardlink vs softlink" thing and making it so it tries hardlink first, and falls back to softlink if that doesn't work. (I.E. not on same filesystem.) Rob 1351644798.0 From prazzb at gmail.com Wed Oct 31 17:46:22 2012 From: prazzb at gmail.com (Prasanna Balan) Date: Wed, 31 Oct 2012 23:16:22 +0530 Subject: [Aboriginal] Breakage while building LFS7.2 Message-ID: <20121031174622.GA7061@desktop> Hello I am trying to build LFS 7.2 using control images.I am using the patch attached below. The build breaks while extracting tzdata.I've attached the ( relevant )build log below the patch. I necessarily added a ls command above mv in the extract function. Notice how the packages above show ls properly and for tzdata it shows up like readlink -f I want to confirm that this is for all and my system is not broken. -- -prazzb GPG key 0xDB63309D -------------- next part -------------- # HG changeset patch # Parent a0645b114a585760839d3c8e82810f8a183ad6ea diff -r a0645b114a58 common/download_functions.sh --- a/common/download_functions.sh Thu Sep 20 14:41:18 2012 +0530 +++ b/common/download_functions.sh Wed Oct 31 23:06:52 2012 +0530 @@ -119,12 +119,14 @@ UNIQUE=$(readlink /proc/self) trap 'rm -rf "$BUILD/temp-'$UNIQUE'"' EXIT rm -rf "$SRCTREE/$PACKAGE" 2>/dev/null - mkdir -p "$BUILD/temp-$UNIQUE" "$SRCTREE" || dienow + mkdir -p "$BUILD/temp-$UNIQUE" "$SRCTREE" || dienow { tar -xv${DECOMPRESS} -f "$FILENAME" -C "$BUILD/temp-$UNIQUE" || dienow } | dotprogress - mv "$BUILD/temp-$UNIQUE/"* "$SRCTREE/$PACKAGE" && + echo $PACKAGE + ls -lh $BUILD/temp-$UNIQUE/* + mv -v "$BUILD/temp-$UNIQUE/"* "$SRCTREE/$PACKAGE" && echo "$SHA1TAR" > "$SHA1FILE" ) diff -r a0645b114a58 images/lfs-bootstrap/download.sh --- a/images/lfs-bootstrap/download.sh Thu Sep 20 14:41:18 2012 +0530 +++ b/images/lfs-bootstrap/download.sh Wed Oct 31 23:06:52 2012 +0530 @@ -1,4 +1,4 @@ -# Build Linux From Scratch 6.7 packages under target environment. +# Build Linux From Scratch 7.2 packages under target environment. # Note: this doesn't rebuild the toolchain packages (libc, binutils, # gcc, linux-headers), but reuses the toolchain we've got, because: @@ -11,8 +11,8 @@ # Download upstream tarball -URL=http://ftp.osuosl.org/pub/lfs/lfs-packages/lfs-packages-6.8.tar \ -SHA1=6ea087c36553a13f4da1152014d226806013db15 \ +URL=http://ftp.osuosl.org/pub/lfs/lfs-packages/lfs-packages-7.2.tar \ +SHA1=eaff8e8a95cb11b6a3199925ed0a87b3fef207a4 \ RENAME='s/-sources//' \ download || dienow @@ -32,22 +32,21 @@ # Fixups for tarball names the Aboriginal extract scripts can't parse into # package name, version number, and archive type. +mv "$SRCDIR"/tzdata{2012e,-2012.5}.tar.gz && mv "$SRCDIR"/expect{,-}5.45.tar.gz && mv "$SRCDIR"/sysvinit-2.88{dsf,}.tar.bz2 && -mv "$SRCDIR"/tcl{8.5.9-src,-src-8.5.9}.tar.gz && -mv "$SRCDIR"/udev-{166-testfiles,testfiles-166}.tar.bz2 || exit 1 +mv "$SRCDIR"/tcl{8.5.12-src,-src-8.5.12}.tar.gz -# Remove damaged patches (whitespace damaged, don't apply without "fuzz"). - -rm "$SRCDIR"/gcc-4.5.2-startfiles_fix-1.patch && -rm "$SRCDIR"/procps-3.2.8-fix_HZ_errors-1.patch || exit 1 -#rm "$SRCDIR"/tar-1.23-overflow_fix-1.patch || exit 1 # Break down upstream tarball for i in $(cd "$SRCDIR"; ls *.tar.*) do - extract_package $(noversion $i) + a=$(noversion $i) + if [ $a != "binutils" -a $a != "bash" -a $a != "linux" -a $a != "glibc" -a $a != "gcc" ] + then + extract_package $a + fi done rm -rf "$SRCDIR" -------------- next part -------------- -rw-r--r-- 1 prazzb users 319K Jul 25 22:08 changes drwxr-xr-x 2 prazzb users 4.0K Jul 25 22:15 compat drwxr-xr-x 2 prazzb users 4.0K Jul 25 22:15 doc drwxr-xr-x 2 prazzb users 4.0K Jul 25 22:15 generic drwxr-xr-x 13 prazzb users 4.0K Jul 25 22:15 library drwxr-xr-x 2 prazzb users 4.0K Jul 25 22:15 libtommath -rw-r--r-- 1 prazzb users 2.3K Jul 16 17:27 license.terms drwxr-xr-x 5 prazzb users 4.0K Jul 25 22:15 macosx drwxr-xr-x 2 prazzb users 4.0K Jul 25 22:15 tests drwxr-xr-x 2 prazzb users 4.0K Jul 25 22:15 tools drwxr-xr-x 3 prazzb users 4.0K Jul 25 22:15 unix drwxr-xr-x 2 prazzb users 4.0K Jul 25 22:15 win ?/home/prazzb/Documents/Mercurial/control-images/build/temp-6964/tcl8.5.12? -> ?/home/prazzb/Documents/Mercurial/control-images/build/lfs-bootstrap/packages/tcl-src? === texinfo ( ) ]2; texinfoExtracting 'texinfo'........................... texinfo total 1.6M -rw-r--r-- 1 prazzb users 75K Dec 26 2007 ABOUT-NLS -rw-r--r-- 1 prazzb users 1.6K Mar 8 2008 AUTHORS -rw-r--r-- 1 prazzb users 35K Jul 2 2007 COPYING -rw-r--r-- 1 prazzb users 275K Sep 19 2008 ChangeLog -rw-r--r-- 1 prazzb users 427K Aug 11 2004 ChangeLog.46 -rw-r--r-- 1 prazzb users 2.2K Jul 13 2007 INSTALL -rw-r--r-- 1 prazzb users 9.2K Jun 15 2006 INSTALL.generic -rw-r--r-- 1 prazzb users 3.9K Apr 11 2004 INTRODUCTION -rw-r--r-- 1 prazzb users 2.5K Jul 6 2008 Makefile.am -rw-r--r-- 1 prazzb users 31K Sep 19 2008 Makefile.in -rw-r--r-- 1 prazzb users 38K Sep 19 2008 NEWS -rw-r--r-- 1 prazzb users 5.4K Dec 13 2004 README -rw-r--r-- 1 prazzb users 2.3K May 22 2008 README.dev -rw-r--r-- 1 prazzb users 12K Jul 27 2008 TODO -rw-r--r-- 1 prazzb users 36K Sep 19 2008 aclocal.m4 drwxr-xr-x 2 prazzb users 4.0K Sep 19 2008 build-aux -rw-r--r-- 1 prazzb users 16K Sep 19 2008 config.in -rwxr-xr-x 1 prazzb users 572K Sep 19 2008 configure -rw-r--r-- 1 prazzb users 7.6K Sep 19 2008 configure.ac drwxr-xr-x 2 prazzb users 4.0K Sep 19 2008 djgpp drwxr-xr-x 3 prazzb users 4.0K Sep 22 2008 doc drwxr-xr-x 4 prazzb users 4.0K Sep 19 2008 gnulib drwxr-xr-x 2 prazzb users 4.0K Sep 19 2008 info drwxr-xr-x 3 prazzb users 4.0K Sep 19 2008 install-info drwxr-xr-x 2 prazzb users 4.0K Sep 19 2008 lib drwxr-xr-x 3 prazzb users 4.0K Sep 19 2008 makeinfo drwxr-xr-x 2 prazzb users 4.0K Sep 19 2008 po -rw-r--r-- 1 prazzb users 7.3K Jul 27 2008 system.h drwxr-xr-x 2 prazzb users 4.0K Sep 19 2008 util ?/home/prazzb/Documents/Mercurial/control-images/build/temp-6997/texinfo-4.13? -> ?/home/prazzb/Documents/Mercurial/control-images/build/lfs-bootstrap/packages/texinfo? === tzdata ( ) ]2; tzdataExtracting 'tzdata' tzdata -rw-r--r-- 1 prazzb users 44K Jul 19 06:15 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/africa -rw-r--r-- 1 prazzb users 16K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/antarctica -rw-r--r-- 1 prazzb users 111K Jul 25 19:43 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/asia -rw-r--r-- 1 prazzb users 67K Jul 25 20:10 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/australasia -rw-r--r-- 1 prazzb users 4.0K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/backward -rw-r--r-- 1 prazzb users 2.9K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/etcetera -rw-r--r-- 1 prazzb users 119K Jul 25 19:43 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/europe -rw-r--r-- 1 prazzb users 393 Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/factory -rw-r--r-- 1 prazzb users 4.3K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/iso3166.tab -rw-r--r-- 1 prazzb users 3.2K Jul 25 19:43 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/leapseconds -rw-r--r-- 1 prazzb users 133K Jul 25 19:43 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/northamerica -rw-r--r-- 1 prazzb users 1.2K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/pacificnew -rw-r--r-- 1 prazzb users 19K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/solar87 -rw-r--r-- 1 prazzb users 19K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/solar88 -rw-r--r-- 1 prazzb users 20K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/solar89 -rw-r--r-- 1 prazzb users 73K Jul 25 19:43 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/southamerica -rw-r--r-- 1 prazzb users 1.6K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/systemv -rw-r--r-- 1 prazzb users 680 Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/yearistype.sh -rw-r--r-- 1 prazzb users 20K Jul 19 06:00 /home/prazzb/Documents/Mercurial/control-images/build/temp-7030/zone.tab mv: target ?/home/prazzb/Documents/Mercurial/control-images/build/lfs-bootstrap/packages/tzdata? is not a directory -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 490 bytes Desc: not available URL: From idunham at lavabit.com Wed Oct 31 22:36:30 2012 From: idunham at lavabit.com (Isaac Dunham) Date: Wed, 31 Oct 2012 15:36:30 -0700 Subject: [Aboriginal] Build from version control system repos instead of from tarballs (Rob Landley) In-Reply-To: References: Message-ID: <20121031153630.eb791f58.idunham@lavabit.com> > On 10/30/2012 03:49:56 AM, Alessio Igor Bogani wrote: > > Hi Rob, > > > > On 30/10/2012 03:36, Rob Landley wrote: > >> On 09/04/2012 03:49:07 AM, Alessio Igor Bogani wrote: > >>> Can Aboriginal grabs sources from git(Linux, musl, uclibc, busybox) > >>> and hg(toybox) repos instead of from tarballs? > >> populating the package cache from the git repo, and > >> Really what I want to do is say "URL=git://busybox.net/busybox" in > >> download.sh, have that clone the relevant repo under packages (so > >> packages/busybox would be a directory), and then just use that as the > >> package cache version. > >> .. > > Personally I don't use tarballs at all anymore: I'll download the > > project's repo and build from it. I'm very happy if Aboriginal let me > > work in this way. > > It currently does, using a two step process: > > 1) Edit "configure" and add the packages you want to do git versions of > to the USE_ALT variable. (USE_ALT=busybox,uClibc,linux) > > 2) Then check out your repos into build/packages/alt-busybox and > build/packages/alt-uClibc and so on. For what it's worth, it would be quite possible to just use "git clone $URL alt-$PACKAGE" > > For example (sorry in git-ish): > > > > git clone aboriginal > > I'm vaguely interested in having a git mirror of the mercurial repo, That would be enough for this purpose, AFAICT. > but I have no interest in hosting any of my projects in git. The git > user interface is appalling, and their solution to all their problems To each his own (I actually enjoy it) > is to expose implementation details to the user. > > cd aboriginal/ && ls > > build.sh config cross-compiler.sh .. (these should works with any > > version and doesn't > > care if gcc is 4.2.2 or 4.7.1) > > The build script required for 4.2 and 4.7 are _totally_different_. > For one thing, newer gcc versions have something like 4 prerequisite > packages you have to build first, requirements 4.2 didn't have. > > Unfortunately, your desire doesn't match reality there. > > > ls package/linux (<- this is a submodule) > > git submodule init > > git submodule update > > cd package/linux > > > git reset --hard origin (reset to the Linus' HEAD) > > (don't change anything other aka let's use Rob's default versions) > > ./build i686-virtio > > How about this: > > I can modify the scripts so that if there's a directory under packages > with the package name, we use that source verbatim. (No patches, no > nothing.) Otherwise, we fall back to the current behavior. (Basically > it's just implementing a search path for the package cache with the > first entry searched before attempting to _populate_ the package cache.) Be sure to make {,dist,real}clean... Otherwise, sounds reasonable to me. > > Why I bother with this crazy request? Because In this way I can > > handle multi-level > > changes in a very simple way. So if I want add a feature which > > needs to change/patch > > kernel, libc and application I could treat this like a single > > atomic commit. Doesn't this make tracking down what caused breakage harder? ie, if you git bisect to an atomic commit in all three, you can't tell if the kernel or the libc patch is responsible. > Which would never go upstream. > > My build uses clean vanilla release versions of upstream packages, and > separates out all the patches (and I try to send those patches upstream > so I don't have to carry them, with varying success). > > This is a design decision, so you can see what I'm using and what I did > to it. > > Your suggestion makes things easier for you at the expense of making it > harder for anybody else to see what you did or reproduce it outside > your build system. That's fine for your purposes, but that's not what > I'm after. Note that this could make GPL conformance (where it matters) a little harder-otherwise, patches plus tarballs are enough, but this may mean you need to make the repo public (I'm not sure how git archive would work with submodules, though) OTOH, if you're using a branch for all your patches, it may make development simpler. > I'm happy to make the build more flexible, and I _want_ my code to to > stay out of your way so you can do what you want to do. I want to make > it trivially easy to overlay "git submodule" on top of the build, but > my build is not becoming dependent on git submodule. -- Isaac Dunham 1351722990.0