From rob at landley.net Wed May 4 09:27:35 2016 From: rob at landley.net (Rob Landley) Date: Wed, 4 May 2016 04:27:35 -0500 Subject: [Aboriginal] Possibly winding down the project. Message-ID: <5729C087.2090302@landley.net> The last-gplv2 toolchains have broken on 3 different architectures recently, all due to kernel changes that didn't regression test on them. Arm broke in linux 4.4, I blogged about it at http://landley.net/notes-2016.html#13-01-2016 and the problem is if you create an empty *.a file, and then link that into a .o file, it hallucinates an arm EABI architecture revision number 0 for the nonexistent code, then linking that into the kernel complains that EABI 0 and EABI version 4 are incompatible. More recent kernels added a flag to say "this isn't real", but said flag is in infrastructure binutils 2.17 doesn't have. This breaks in the 4.4 kernel because they unconditionally included a virtualization infrastructure subdirectory, which builds even when the config symbol is switched off. Mips broke (see http://landley.net/notes-2016.html#15-02-2016) because they unconditionally enabled VDSO which requires binutils 2.23. While they do HAVE a probe for older binutils versions, it has assumptions about how many digits wide your version fields are that break on the git snapshot I'm using which identifies itself as "2.17.50.20070703" and when I fix THAT, the assembly for the non-VDSO #else case still uses assembly language constructs that weren't in binutils 2.17. Now sh4 broke with a segfault building crypto/because kernel commit 798b2cbf9 unconditionally sets CRYPTO if you enable IPV4, and the crypto menu inherits 22 forced on symbols from that including crap like 3DES that nobody sane uses anymore, and one of those now does: CC crypto/drbg.o crypto/drbg.c: In function 'drbg_kcapi_random': crypto/drbg.c:1729: internal compiler error: Segmentation fault Please submit a full bug report, So, wheee. There's been a GPLv3 toolchain submission here, which I admit is half of what's discouraging me from working on aboriginal linux anymore. Putting effort into my old toolchain... people don't want to use it so what's the point? Rich is doing a new GPLv3 toolchain at https://github.com/richfelker/musl-cross make and good luck with that too. But I will never distribute a binary GPLv3 program in a hobbyist context. If a corporation wants to stand up and take the legal liability and pay me to do it on their behalf as an employee so THEY are the one getting sued, fine. But I'm not doing it as me ever. (Keep in mind I outwaited AOL and Windows, and am in the process of outwaiting facebook. The sun only has so much hydrogen, I am waiting for GPLv3 to go away and working to build public domain alternatives to it.) Defending GPLv2 against GPLv3 was always a stopgap measure, the proper fix is for GPL to go away entirely and be replaced by licensing closer to the public domain. (I.E. the FSF probably outlive Richard Stallman, and the best thing I can do is work towards obsoleting them.) I've been playing a little with LLVM, maybe I should do more of that, but it's a giant hairball nightmare written in C++ and I don't really WANT to have anything to do with it. The only advantage is it's not FSF-maintained GPLv3 crap. I still need to use this to test toybox, but to be honest I can stay on linux 4.3 for years doing that. Nobody makes binary native toolchains, but maybe I can make puppy eyes at Rich about that, and if not... I've got my old ones building under my old kernel and when I'm ready to open the AOSP can of worms they build under LLVM anyway. Bootstrapping debian would be nice, but I'm not sure anybody under the age of 40 still cares about non-Android Linux? Heck, even most of the android developers are using windows and mac hosts. No seriously, there's numbers: http://stackoverflow.com/research/developer-survey-2015#tech-os Sigh. I _can_ fix everything that's currently wrong with the old toolchains (although it's probably a couple weeks of debugging, that's just what I _know_ about). Or I could ask Rich to do native toolchains, make a really small shell script to build and paclage a toybox userspace, and move on with my life. I'm sticking on Ubuntu 14.04 (last non-systemd LTS version) until it's end of lifed, which gives me a deadline to get a working self-hosting android workstation working... Anybody else care to venture an opinion? Rob From rob at landley.net Wed May 4 09:52:57 2016 From: rob at landley.net (Rob Landley) Date: Wed, 4 May 2016 04:52:57 -0500 Subject: [Aboriginal] Possibly winding down the project. In-Reply-To: <5729C087.2090302@landley.net> References: <5729C087.2090302@landley.net> Message-ID: <5729C679.7070802@landley.net> On 05/04/2016 04:27 AM, Rob Landley wrote: > I've been playing a little with LLVM, maybe I should do more of that, > but it's a giant hairball nightmare written in C++ and I don't really > WANT to have anything to do with it. The only advantage is it's not > FSF-maintained GPLv3 crap. The big problem with llvm is that they're moving from autoconf to cmake, which means the snapshotted configure file in the project that lets you NOT install automake on the host is going away, replaced by something that depends on python at runtime. So llvm is growing python as a build-time prerequisite, and they don't see this as a bad thing. Maybe someday I'll have time to do qcc, maybe someday pcc will lurch back from the dead, or maybe somebody will write a C port of cmake. But a toolchain that requires python as a hard build-time prerequisite is not part of a sane system bootstrap. Rob (I'm aware that ditching autoconf and gmake is a laudable goal. But cmake isn't it.) From paul.sherwood at codethink.co.uk Wed May 4 14:35:13 2016 From: paul.sherwood at codethink.co.uk (Paul Sherwood) Date: Wed, 04 May 2016 15:35:13 +0100 Subject: [Aboriginal] Possibly winding down the project. In-Reply-To: <5729C087.2090302@landley.net> References: <5729C087.2090302@landley.net> Message-ID: <1f5fe5e48c75f0faa74bd6762e24d0c7@codethink.co.uk> Hi Rob, On 2016-05-04 10:27, Rob Landley wrote: > The last-gplv2 toolchains have broken on 3 different architectures > recently, all due to kernel changes that didn't regression test on > them. Gah! > There's been a GPLv3 toolchain submission here, which I admit is half > of > what's discouraging me from working on aboriginal linux anymore. > Putting > effort into my old toolchain... people don't want to use it so what's > the point? For better or worse, Codethink sponsored the GPLv3 toolchain work. Please do not be discouraged. Note that: - we are very interested in Aboriginal - it's great work! - we (as well as some of our customers) recognise the need to maintain permissively licensed, boot-strappable toolchain software over the long term - we would have preferred to build on your established GPLv2 approach, but as you say, various upstreams are now breaking... which will ultimately close that path. > Rich is doing a new GPLv3 toolchain at > https://github.com/richfelker/musl-cross make and good luck with that > too. > > But I will never distribute a binary GPLv3 program in a hobbyist > context. If a corporation wants to stand up and take the legal > liability > and pay me to do it on their behalf as an employee so THEY are the > one > getting sued, fine. But I'm not doing it as me ever. (Keep in mind I > outwaited AOL and Windows, and am in the process of outwaiting > facebook. > The sun only has so much hydrogen, I am waiting for GPLv3 to go away > and > working to build public domain alternatives to it.) > > Defending GPLv2 against GPLv3 was always a stopgap measure, the > proper > fix is for GPL to go away entirely and be replaced by licensing > closer > to the public domain. (I.E. the FSF probably outlive Richard > Stallman, > and the best thing I can do is work towards obsoleting them.) > > I've been playing a little with LLVM, maybe I should do more of that, > but it's a giant hairball nightmare written in C++ and I don't > really > WANT to have anything to do with it. The only advantage is it's not > FSF-maintained GPLv3 crap. > > I still need to use this to test toybox, but to be honest I can stay > on > linux 4.3 for years doing that. Nobody makes binary native > toolchains, > but maybe I can make puppy eyes at Rich about that, and if not... > I've > got my old ones building under my old kernel and when I'm ready to > open > the AOSP can of worms they build under LLVM anyway. > > Bootstrapping debian would be nice, but I'm not sure anybody under > the > age of 40 still cares about non-Android Linux? Heck, even most of the > android developers are using windows and mac hosts. No seriously, > there's numbers: > > http://stackoverflow.com/research/developer-survey-2015#tech-os I'm not sure winning the popularity contests matters much, really :) In any case, let's avoid monoculture and encourage diversity. > Sigh. I _can_ fix everything that's currently wrong with the old > toolchains (although it's probably a couple weeks of debugging, > that's > just what I _know_ about). Or I could ask Rich to do native > toolchains, > make a really small shell script to build and paclage a toybox > userspace, and move on with my life. I'm sticking on Ubuntu 14.04 > (last > non-systemd LTS version) until it's end of lifed, which gives me a > deadline to get a working self-hosting android workstation working... > > Anybody else care to venture an opinion? For gcc+binutils, the tide seems to be against us. While android is fine for various scenarios, we'd prefer to support a route with a broader sponsor base. Would you be interested in collaboration to move forward with Aboriginal on all possible architectures using clang + musl? br Paul From alanpost at sunflowerriver.org Wed May 4 15:04:21 2016 From: alanpost at sunflowerriver.org (Alan Post) Date: Wed, 4 May 2016 09:04:21 -0600 Subject: [Aboriginal] Possibly winding down the project. In-Reply-To: <1f5fe5e48c75f0faa74bd6762e24d0c7@codethink.co.uk> References: <5729C087.2090302@landley.net> <1f5fe5e48c75f0faa74bd6762e24d0c7@codethink.co.uk> Message-ID: <20160504150421.GC5409@sunflowerriver.org> On Wed, May 04, 2016 at 03:35:13PM +0100, Paul Sherwood wrote: > >Sigh. I _can_ fix everything that's currently wrong with the old > >toolchains (although it's probably a couple weeks of debugging, > >that's > >just what I _know_ about). Or I could ask Rich to do native > >toolchains, > >make a really small shell script to build and paclage a toybox > >userspace, and move on with my life. I'm sticking on Ubuntu 14.04 > >(last > >non-systemd LTS version) until it's end of lifed, which gives me a > >deadline to get a working self-hosting android workstation working... > > > >Anybody else care to venture an opinion? > > For gcc+binutils, the tide seems to be against us. > > While android is fine for various scenarios, we'd prefer to support > a route with a broader sponsor base. > > Would you be interested in collaboration to move forward with > Aboriginal on all possible architectures using clang + musl? > Hello Rob, I've submitted a bug report and a minor patch to you over the years. I find Aboriginal tremendously useful so I'll weigh in. My use case for Aboriginal is building binaries that I can distribute across Centos/RHEL, Debian/Ubuntu, and SUSE. The OpenSSL fiasco got me serious about rebuilding dependencies, which then exposed problems in the age of my toolchain. Paul's clang + musl proposal is more-or-less where I see myself going -- for all the reasons discussed in this thread. I already have to build cmake for other packages I depend on so that is only regular pain rather than acute pain. I'm interested in Paul's idea. -A From rob at landley.net Thu May 5 00:49:55 2016 From: rob at landley.net (Rob Landley) Date: Wed, 4 May 2016 19:49:55 -0500 Subject: [Aboriginal] Possibly winding down the project. In-Reply-To: <20160504150421.GC5409@sunflowerriver.org> References: <5729C087.2090302@landley.net> <1f5fe5e48c75f0faa74bd6762e24d0c7@codethink.co.uk> <20160504150421.GC5409@sunflowerriver.org> Message-ID: <572A98B3.508@landley.net> On 05/04/2016 10:04 AM, Alan Post wrote: > On Wed, May 04, 2016 at 03:35:13PM +0100, Paul Sherwood wrote: > >>> Sigh. I _can_ fix everything that's currently wrong with the old >>> toolchains (although it's probably a couple weeks of debugging, >>> that's >>> just what I _know_ about). Or I could ask Rich to do native >>> toolchains, >>> make a really small shell script to build and paclage a toybox >>> userspace, and move on with my life. I'm sticking on Ubuntu 14.04 >>> (last >>> non-systemd LTS version) until it's end of lifed, which gives me a >>> deadline to get a working self-hosting android workstation working... >>> >>> Anybody else care to venture an opinion? >> >> For gcc+binutils, the tide seems to be against us. >> >> While android is fine for various scenarios, we'd prefer to support >> a route with a broader sponsor base. >> >> Would you be interested in collaboration to move forward with >> Aboriginal on all possible architectures using clang + musl? >> > > Hello Rob, I've submitted a bug report and a minor patch to you > over the years. I find Aboriginal tremendously useful so I'll > weigh in. > > My use case for Aboriginal is building binaries that I can > distribute across Centos/RHEL, Debian/Ubuntu, and SUSE. The > OpenSSL fiasco got me serious about rebuilding dependencies, > which then exposed problems in the age of my toolchain. > > Paul's clang + musl proposal is more-or-less where I see myself > going -- for all the reasons discussed in this thread. I already > have to build cmake for other packages I depend on so that is only > regular pain rather than acute pain. > > I'm interested in Paul's idea. I'm poking at an llvm build according to the BLFS instructions, but leaving it to run for hours it still hasn't completed. It takes longer to build than the a whole aboriginal target did (cross toolchain, native toolchain, root filesystem, and kernel combined). (And yes, I've told it --enable-targets=x86_64 so at least this time I don't come back in the morning to find it building powerpc stuff.) I'm much less discouraged about llvm since learning that cmake doesn't seem to require a python build prequisite (or at least there's a version of it that doesn't), but it's a serious pig to build isn't it? Still, maybe that's a way forward. Doesn't support all the targets yet, but... Rob From sam.thursfield at codethink.co.uk Thu May 5 08:02:41 2016 From: sam.thursfield at codethink.co.uk (Sam Thursfield) Date: Thu, 5 May 2016 10:02:41 +0200 Subject: [Aboriginal] Possibly winding down the project. In-Reply-To: <572A98B3.508@landley.net> References: <5729C087.2090302@landley.net> <1f5fe5e48c75f0faa74bd6762e24d0c7@codethink.co.uk> <20160504150421.GC5409@sunflowerriver.org> <572A98B3.508@landley.net> Message-ID: <572AFE21.3000501@codethink.co.uk> On 05/05/16 02:49, Rob Landley wrote: > I'm poking at an llvm build according to the BLFS instructions, but > leaving it to run for hours it still hasn't completed. It takes longer > to build than the a whole aboriginal target did (cross toolchain, native > toolchain, root filesystem, and kernel combined). (And yes, I've told it > --enable-targets=x86_64 so at least this time I don't come back in the > morning to find it building powerpc stuff.) > > I'm much less discouraged about llvm since learning that cmake doesn't > seem to require a python build prequisite (or at least there's a version > of it that doesn't), but it's a serious pig to build isn't it? Actually a design goal of CMake is to require a C++ compiler and little else to build it. They target Windows, where every extra dependency comes at quite a cost. Sam -- Sam Thursfield, Codethink Ltd. Office telephone: +44 161 236 5575 From james.w.mcmechan at gmail.com Sat May 28 18:20:57 2016 From: james.w.mcmechan at gmail.com (James McMechan) Date: Sat, 28 May 2016 11:20:57 -0700 Subject: [Aboriginal] how to use BUILD again Message-ID: I was playing around and did get the BUILD variable to work. it appears that the export in config must have a full path for it to work correctly e.g. export BUILD=/home/mcmechan/Downloads/aboriginal-1.4.5/jim or export BUILD=$PWD/jim will build all of the files under ./jim rather than ./build For some reason I am not quite understanding how the interaction between the two locations where build is set - is supposed to work. sources/variables.sh:export_if_blank BUILD="$TOP/build" build.sh: [ -z "$BUILD" ] && BUILD="build" but the config file does not have $TOP set so I ending up using $PWD instead rather than hardcodeing the full path. Is build.sh supposed to source sources/variables.sh after sourceing config? or source sources/include.sh? At the moment BUILD is set two different ways in two different files so it may be a future maintenance issue. Anyway it is working for me. Jim McMechan -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob at landley.net Sun May 29 03:10:58 2016 From: rob at landley.net (Rob Landley) Date: Sat, 28 May 2016 22:10:58 -0500 Subject: [Aboriginal] how to use BUILD again In-Reply-To: References: Message-ID: <574A5DC2.3080109@landley.net> On 05/28/2016 01:20 PM, James McMechan wrote: > I was playing around and did get the BUILD variable to work. > it appears that the export in config must have a full path for it to > work correctly Yes, did I forget to document that? > e.g. > export BUILD=/home/mcmechan/Downloads/aboriginal-1.4.5/jim > or > export BUILD=$PWD/jim > will build all of the files under ./jim rather than ./build I haven't regression tested that in forever. Hmmm, I can readlink -f a path at the start of a script to absolutize it, but these days I'm leaning towards trying to REMOVE complexity instead of add it. For example, I created a really simple root filesystem builder script for the j-core project: http://lists.j-core.org/pipermail/j-core/2016-May/000171.html I'm halfway towards just gritting my teeth and fixing the three broken architectures I'm aware of. Only two of which are broken by the _next_ kernel, but I'm like 3 releases behind now. Sigh, I can defend the old toolchain if I keep on top of it, I'd just really LIKE llvm to start mattering at some point. > For some reason I am not quite understanding how the interaction between > the two locations where build is set - is supposed to work. > > sources/variables.sh:export_if_blank > BUILD="$TOP/build" $TOP is $PWD when you first run the build. I only snapshot it because you can cd around (each setupfor does that, for example) and then need to refer to it again. > build.sh: [ -z "$BUILD" ] && BUILD="build" Yeah, I really shouldn't be doing that twice. The problem is build.sh doesn't pull in sources/include.sh (which includes the other files in sources) because it _should_ be a trivial wrapper script without needing all the infrastructure the rest of the build has. (I also don't have the double-inclusion guards, but it wouldn't be hard to add those. My real concern is that if ./root-filesystem.sh was always run from build.sh it could grow a dependency that wouldn't get regularly checked.) But then build.sh needed to refer to build/ to check if the tarballs were already there (which is how the stage-level build dependencies work. I've done more granular dependencies for host-tools.sh and more recently for system-image.sh, but not for the other stages). So I supplied a default value there. The compromise I made was NOT exporting it, so the child scripts could overwrite it. Which is sad because it's not pulling from "config"... > but the config file does not have $TOP set so I ending up using $PWD > instead rather than hardcodeing the full path. Using $PWD in build.sh is fine because we never cd in that script, only in child processes with their own environment. > Is build.sh supposed to source sources/variables.sh after sourceing > config? or source sources/include.sh? No, it's intentionally not doing that because then the individual scripts sourcing it would be a NOP and dependency leakage without sufficient regression testing would be an issue. (Which $BUILD itself _is_, but that one I needed to avoid rebuilding everything every time. I could have native-compiler.sh check if the tarball is there, but I _want_ it to force a rebuild if called outside build.sh...) > At the moment BUILD is set two different ways in two different files so > it may be a future maintenance issue. > > Anyway it is working for me. I'm glad. I should update the docs... > Jim McMechan Rob From cbaumann at visteon.com Mon May 30 07:00:50 2016 From: cbaumann at visteon.com (Baumann, Christoph (C.)) Date: Mon, 30 May 2016 07:00:50 +0000 Subject: [Aboriginal] Aboriginal for Cortex-A15 Message-ID: Hello! As currently Aboriginal has no target definition for ARMv7 (Cortex-A15 to be more precise), I started setting it up myself. Compilation succeeds, but so far the resulting image wouldn't boot at all. QEMU crashes even before trying to start the kernel: "PFLASH: Possible BUG - Write block confirm". The qemu command used is this: qemu-system-arm -m 1024M -M vexpress-a15 -serial stdio -cpu cortex-a15 -nographic -no-reboot -kernel linux -initrd rootfs.cpio.gz -append "mem=1024M panic=1 HOST=armv7lhf $KERNEL_EXTRA" $QEMU_EXTRA As kernel config I used arch/arm/configs/vexpress_defconfig. Any hints or ideas? Regards, Christoph From rob at landley.net Mon May 30 21:59:06 2016 From: rob at landley.net (Rob Landley) Date: Mon, 30 May 2016 16:59:06 -0500 Subject: [Aboriginal] Aboriginal for Cortex-A15 In-Reply-To: References: Message-ID: <574CB7AA.9030804@landley.net> On 05/30/2016 02:00 AM, Baumann, Christoph (C.) wrote: > Hello! > > As currently Aboriginal has no target definition for ARMv7 (Cortex-A15 to be more precise), The toolchain we're using is the last GPLv2 release, and that only had armv6. We updated the binutils to the last version out of git before the license switch, which does have armv7 support, but the kernel build gets confused. I hope to move the project to LLVM, but it's not quite ripe yet. > I started setting it up myself. > Compilation succeeds, but so far the resulting image wouldn't boot at all. > QEMU crashes even before trying to start the kernel: "PFLASH: Possible BUG > - Write block confirm". > The qemu command used is this: > qemu-system-arm -m 1024M -M vexpress-a15 -serial stdio -cpu cortex-a15 -nographic -no-reboot -kernel linux -initrd rootfs.cpio.gz -append "mem=1024M panic=1 HOST=armv7lhf $KERNEL_EXTRA" $QEMU_EXTRA > > As kernel config I used arch/arm/configs/vexpress_defconfig. I haven't tried that one, but in general the kernel's gcc feature probes of the old toolchain's capabilities and what features it tries to select don't agree. (And sometims fail silently.) > Any hints or ideas? Tristan Van Berkholm has a fork with GPLv3 licensed gcc/binutils versions: http://lists.landley.net/htdig.cgi/aboriginal-landley.net/2016-February/001584.html I also note that you can build an armv5l or armv6l root filesystem and use a different toolchain to build your kernel. (I have qemu using the external initramfs load support by default, so building cpio and vmlinux separately should work fine.) Rob