From rob at landley.net Wed Aug 1 11:12:38 2012 From: rob at landley.net (Rob Landley) Date: Wed, 01 Aug 2012 06:12:38 -0500 Subject: [Aboriginal] Fwd: greetings! In-Reply-To: References: <4FFB9E95.9050302@landley.net> <4FFDACF5.9050108@landley.net> <50076C48.5010403@landley.net> <501044B6.5010504@landley.net> Message-ID: <50190F26.5000405@landley.net> On 07/26/2012 08:17 AM, David Henderson wrote: > This email thread has started to become soup! So... I'll start a new > paragraph so we can start over. :) > > Going back to the software included in the "firmware", the list should > be very small and only allow the device to get to the point for user > interaction. For example, > > - busybox (later toybox) for core functionality including user > management, scheduling, etc > - bash > - clAPI (our own software) > - the appropriate mount binaries for ext, ntfs, nfs, and smb/cifs Once upon a time I wrote the busybox mount. (Well, rewrote it 3 times until there was nothing left of the original.) smb and nfs shouldn't need mount binaries, you can mount them with the right -o command line. All the mount binary would do is prompt you for password so you don't have to say pass= in the -o line in plaintext. ext? shouldn't need a special mount binary, just the standard one and the kernel module (if it's not static). ntfs I'm not familiar with. > - ssh I've got static dropbear binaries up at http://landley.net/aboriginal/bin natively built by a control-image. > - ssl (for secured headless devices or other communication) How do you get anything else to use it? Dropbear is self-contained the busybox web server and wget don't link against eternal libraries. (I thought about makign them use stunnel but never got around to it. Did that happen in my absence?) > - ssmtp (or similar) for automated outbound communication of device busybox has had a "sendmail" command for a while now. > * optional alsa (for sound enabled devices) > * optional graphics software for headed devices (headless will already > be handled via clAPI) Those were always very device-specific to me, but I've never understood the various incarnations of the linux sound layer all that well (just enough to really hate pulseaudio), and the phrase "graphics software for headless devices" reads like "hats for the headless horseman"... > Items such as mplayer, samba, rdesktop, okular, perl, rsync, etc will be > outside the scope of the "firmware" because they aren't required to turn > on a device, login, and get to either the prompt or blank desktop > (GUI). The list above should enable a device to communicate/interact > with other devices on the most basic level, but nothing beyond. > Because the "firmware" is kept to an absolute minimum, I think any > library API changes will not really disrupt the device from working. Well, good luck with it. > Regarding the musl switch over, I had no clue that uClibc has been > shelved for so long! Yeah, it got run over by buildroot as I explained here: http://landley.net/aboriginal/history.html (I _really_ need to finish that page at some point.) In theory uClibc development is unstuck now but its baseline development rate is way way way slower than it used to be, because the project is bloated and has decades of history nobody really seems to understand anymore. Here's the start of a long "uClibc has a problem" thread from 2007 talking about stagnation _then_ which wasn't fixed for another 3-4 years: http://lists.uclibc.org/pipermail/uclibc/2007-September/039215.html I started sending the maintainer a cake when it had been a year between releases back in 2005: http://lists.uclibc.org/pipermail/uclibc/2005-January/010877.html http://lists.uclibc.org/pipermail/uclibc/2006-December/037750.html Here's the 2006 OLS paper on NPTL support for uClibc: http://kernel.org/doc/ols/2006/ols2006v1-pages-409-420.pdf It took ~5 years for that to actually get merged and supported on all targets. What happened was sjhill refused to merge his work until he was paid for it, and other people got into the habit of doing extensive work in their own private forks. The maintainer had mostly wandered off to do other things (buildroot, his own business) and didn't police this, so it got to the point where they were actively driving _away_ anybody who committed to the main fork (because it made them do extra work merging stuff into their private forks). Towards the end of 2008 I appointed a new uClibc maintainer more or less by fiat to try to fix this: http://lists.uclibc.org/pipermail/uclibc/2008-October/041191.html But they didn't get NPTL support into all architectures and actually working for something like two years after that. Basically the tree was such a horrible mess at that point, with at least _three_ independent NPTL implementations for various architectures (because sjhill wouldn't show anybody his code), that cleanup work took a long time. Development finally got moving forward again under that new maintainer a couple years ago, but it's very slow and there's a huge amount of inertia. The eglibc project arose because of the vacuum created by uClibc's stagnation, for example. The upshot is that uClibc is no longer _simple_, and the "first mostly works" version (0.9.24) was in 2003, and was the _eighth_ release that year. There were then 2 releases in 2004, one in 2005, and NONE in the whole of 2006. (Hence the sending of cakes.) The project still hasn't recovered from that. > Why wouldn't the switch to musl be more critical? musl currently supports x86, x86-64, and arm. I currently have mips, sh4, sparc, and powerpc working under uClibc. There's a lot more potential in musl, but it's not fleshed out yet. > Not only would you get actively developed software, but the speed > increases as shown in one of the links below. I'm not doing it to speed up code, I'm doing it so that if I post a bug report to the mailing list (and follow up repeatedly) it won't get ignored for a year and a half. I'd also like to switch to a libc implementation that hasn't got THREE separate threading implementations. http://lists.busybox.net/pipermail/uclibc/2008-September/041023.html And one where x86-64 support isn't a buggy afterthought: http://lists.uclibc.org/pipermail/uclibc/2011-October/045835.html (And no, glibc isn't even in consideration. It requires perl to build.) > I can see your point about standardization in the qwerty keyboard > example provided, however, because of how Linux has evolved compared to > the design goals of the XiniX distro, it's easier to manage the OS by > separating it as I have. Otherwise, there would be an unwieldy amount > of mounts to maintain with software and configs going in various places > (e.g. /bin, /usr/bin, /usr/local/bin, /usr/share, etc). The layout for > XiniX is uniform and makes sense where things are located and why. Good luck reinventing the wheel. > Thanks for all the articles on the GPLv2 vs GPLv3 - lots of reading to > do! I'm sure I'll have more questions for you once I'm done. :) I need to write up a unified explanation of that. I need to finish the history page I linked to earlier. I need to write up the git-bisect-howto follow up I've had half-finished for 3 months. I need to catch up on my email... Rob -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1343819558.0 From rob at landley.net Wed Aug 1 11:50:14 2012 From: rob at landley.net (Rob Landley) Date: Wed, 01 Aug 2012 06:50:14 -0500 Subject: [Aboriginal] [Qemu-devel] [PATCH] Fixing sh4 serial abort In-Reply-To: References: <50129B8A.3060102@landley.net> <5012CCD5.7020205@landley.net> Message-ID: <501917F6.1040304@landley.net> On 07/27/2012 12:28 PM, Peter Maydell wrote: > On 27 July 2012 18:16, Rob Landley wrote: >> On 07/27/2012 09:32 AM, Peter Maydell wrote: >>> The thing this analysis is missing is any examination of the question >>> "what is the hardware we are modelling documented to do?". >> >> Given that 3.3, 3.4, and 3.5 kernels have already shipped with this, I'm >> guessing "not immediately crash"? > > I said "documented", not "what it happens to do in practice". I'm trying to make it work in practice. >> So the arm maintainer noticed a place where the code didn't match the >> documentation, changed the code to match the docs, and result doesn't >> work under qemu's versatile board emulation. But at least 3.5 doesn't >> work for a _different_ reason than 3.4 didn't work, so there's that. > > PCI on versatilepb is hopelessly broken -- the only thing the kernel > code does (or did) is work on QEMU. Which is all I used it for. >> I hadn't reported this one yet because I still haven't root caused it, >> just bisected to find the break. I know reverting the IRQ assignment >> line in 3.5 doesn't fix it, which implies the "swizzle" bit is to blame >> (which seems ot have something to do with PCI bridges), and thus calling >> the default function instead of calling no function breaks qemu's >> versatile board emulation. > > 1. Find Arnd's kernel patches and see what of them still need to > be applied (http://comments.gmane.org/gmane.linux.ports.arm.kernel/93393) > 2. Get kernel working on real hardware You seem to be under the impression I have real arm hardware. (Or real sh4 hardware, or real powerpc hardware, or real mips hardware...) I've got an armv4t board in a box (a patch to support that was submited to the list by the vendor, and never merged), and a phone that's presumably armv7l but it's in use as a phone with the vendor software on it, and at work I use an cluster of unreleased armv7 SOCs in an "I can't believe it's not NUMA" configuration. The userspaces I worked out under qemu run in a chroot under there. I've got two diferent mips routers in a box somewhere with a dongle that lets me do a serial console but both require kernel patches to add board support (neither vendor ever submitted it upstream). I've run the root filesystems I worked out under qemu in a chroot under them but haven't gotten them out of the box in over a year. (I think it's in storage, actually.) A couple of the game consoles might be powerpc, but I've never run linux on them. I don't own sh4 or sparc hardware. > 3. Submit qemu patches to fix our versatilepb PCI emulation to > match hardware 4) locally patch the kernel back to work with existing qemu releases since all I want to do is boot an arm userspace to a shell prompt and run arm code on it, and don't really care about the board emulation except for the laundry list of features I need out of it such as "a working persistent clock so make doesn't lose its marbles", "enough physical memory", "serial console", "working network device", and "three working block device I can plug filesystem images into". My goal is to run an emulated linux userspace, and application emulation is _way_ more complicated and fiddly than system emulation. If real hardware was as easily accessible for me as qemu, qemu would be significantly less useful. > If you like you can do 3 first and I'll happily apply those patches > regardless of whether anybody cares to fix the kernel. I just want it to work. Working the same way real hardware does is a bonus, but if the change isn't visible to userspace (virtio? emulated scsi? emulated ide?) it's not actually relevant to my use case. > I'm afraid I don't have a great deal of time for versatilepb > emulation fixes because it's an incredibly ancient devboard. I can switch to a newer board, but I want to plug armv4tl, armv5l, armv6l, and armv7l processors into it. (And eventually armv8 but nothing supports that yet.) If it's always running armv7 then I can't _prove_ that this userspace would actually run on armv5 and didn't leak armv7 instructions in there. Last time I looked at newer boards the idea of plugging older processors into them confused both qemu and the kernel's config stuff. Looking at current qemu-git, there's no "-M vexpress", there is instead vexpress-a9 ARM Versatile Express for Cortex-A9 vexpress-a15 ARM Versatile Express for Cortex-A15 I.E. the assumption about what processor you're plugging into this board is baked into the board _name_, and both of those are armv7 only. Hmmm, then again it looks like http://www.mail-archive.com/qemu-devel at nongnu.org/msg19370.html Never got merged so the -cpu restrictions for armv4t and armv5l are currently commented out anyway. (I tested with it at one point, but I ship system images that people use with their existing qemu versions, so I can't patch qemu for them....) > vexpress breakage will get more attention from me (though I > don't habitually test new kernels on qemu so I won't necessarily > notice bugs unless my attention is drawn to them.) I do habitually test newer kernels on qemu. I can draw attention to them, but not necessarily interest. > -- PMM I'll poke at vexpress and see what I can get it to do. Rob -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1343821814.0 From peter.maydell at linaro.org Wed Aug 1 12:01:36 2012 From: peter.maydell at linaro.org (Peter Maydell) Date: Wed, 1 Aug 2012 13:01:36 +0100 Subject: [Aboriginal] [Qemu-devel] [PATCH] Fixing sh4 serial abort In-Reply-To: <501917F6.1040304@landley.net> References: <50129B8A.3060102@landley.net> <5012CCD5.7020205@landley.net> <501917F6.1040304@landley.net> Message-ID: On 1 August 2012 12:50, Rob Landley wrote: > I can switch to a newer board, but I want to plug armv4tl, armv5l, > armv6l, and armv7l processors into it. (And eventually armv8 but nothing > supports that yet.) If it's always running armv7 then I can't _prove_ > that this userspace would actually run on armv5 and didn't leak armv7 > instructions in there. > > Last time I looked at newer boards the idea of plugging older processors > into them confused both qemu and the kernel's config stuff. Yes, this is because it fundamentally doesn't work and won't work. ARM board models work with a limited set of CPUs (often just one CPU), if you try to -cpu something into a board model that doesn't support it you get to keep both pieces when it breaks. You can't just plug an A9 CPU into a versatile PB board in hardware, and it doesn't work in QEMU for about the same reason (the set of connections between the CPU and the board is completely different). > Looking at current qemu-git, there's no "-M vexpress", there is instead > > vexpress-a9 ARM Versatile Express for Cortex-A9 > vexpress-a15 ARM Versatile Express for Cortex-A15 > > I.E. the assumption about what processor you're plugging into this board > is baked into the board _name_, and both of those are armv7 only. This is because the A9 and A15 versatile express systems are genuinely different hardware (the memory maps are all different, for example). > Hmmm, then again it looks like > > http://www.mail-archive.com/qemu-devel at nongnu.org/msg19370.html > > Never got merged so the -cpu restrictions for armv4t and armv5l are > currently commented out anyway. I think we subsequently merged a different patch for the equivalent feature. Certainly translate.c does attempt to check for ARCH(5) or ARCH(4T) and so on. If we get specific cases wrong I can fix them. -- PMM 1343822496.0 From dhenderson at digital-pipe.com Thu Aug 2 14:52:50 2012 From: dhenderson at digital-pipe.com (David Henderson) Date: Thu, 2 Aug 2012 10:52:50 -0400 Subject: [Aboriginal] Fwd: greetings! In-Reply-To: <50190F26.5000405@landley.net> References: <4FFB9E95.9050302@landley.net> <4FFDACF5.9050108@landley.net> <50076C48.5010403@landley.net> <501044B6.5010504@landley.net> <50190F26.5000405@landley.net> Message-ID: On Wed, Aug 1, 2012 at 7:12 AM, Rob Landley wrote: > On 07/26/2012 08:17 AM, David Henderson wrote: > > This email thread has started to become soup! So... I'll start a new > > paragraph so we can start over. :) > > > > Going back to the software included in the "firmware", the list should > > be very small and only allow the device to get to the point for user > > interaction. For example, > > > > - busybox (later toybox) for core functionality including user > > management, scheduling, etc > > - bash > > - clAPI (our own software) > > - the appropriate mount binaries for ext, ntfs, nfs, and smb/cifs > > Once upon a time I wrote the busybox mount. (Well, rewrote it 3 times > until there was nothing left of the original.) > > smb and nfs shouldn't need mount binaries, you can mount them with the > right -o command line. All the mount binary would do is prompt you for > password so you don't have to say pass= in the -o line in plaintext. > > ext? shouldn't need a special mount binary, just the standard one and > the kernel module (if it's not static). > > ntfs I'm not familiar with. > I did a quick look at the busybox mount help, but didn't see any -o parameters for specifying certain values (e.g. user,pass,dom,etc) to mount network shares. Perhaps some testing is in order! :) The XiniX "firmware" already has several external mount binaries included (mount.cifs and mount.ntfs), so it shouldn't be a problem either way. :) > > > - ssh > > I've got static dropbear binaries up at > http://landley.net/aboriginal/bin natively built by a control-image. > Have you run into any problems with dropbear over openSSH? Aside from the small size, are there any benefits of using dropbear? > > > - ssl (for secured headless devices or other communication) > > How do you get anything else to use it? Dropbear is self-contained the > busybox web server and wget don't link against eternal libraries. (I > thought about makign them use stunnel but never got around to it. Did > that happen in my absence?) > Shouldn't this be installed for (optional) secured headless web interfaces? As an alternative to wget, curl can be used (which can be used with openSSL). Any advantage either way? > > > - ssmtp (or similar) for automated outbound communication of device > > busybox has had a "sendmail" command for a while now. > How easy is it to get the BB sendmail working? I was able to get sSMTP up and running in a few minutes... > > > * optional alsa (for sound enabled devices) > > * optional graphics software for headed devices (headless will already > > be handled via clAPI) > > Those were always very device-specific to me, but I've never understood > the various incarnations of the linux sound layer all that well (just > enough to really hate pulseaudio), and the phrase "graphics software for > headless devices" reads like "hats for the headless horseman"... > Yes, the "graphics software for *headed* devices" is somewhat general, but will include things like GTK, QT, Awesome WM, etc). This area is device-specific, but can be setup easily via the installation of the "firmware" for the device. > > > Items such as mplayer, samba, rdesktop, okular, perl, rsync, etc will be > > outside the scope of the "firmware" because they aren't required to turn > > on a device, login, and get to either the prompt or blank desktop > > (GUI). The list above should enable a device to communicate/interact > > with other devices on the most basic level, but nothing beyond. > > Because the "firmware" is kept to an absolute minimum, I think any > > library API changes will not really disrupt the device from working. > > Well, good luck with it. > I take it you don't think that will be the case. :) How many libraries will be used with the basic set of software listed above? It should be far less than a typical distro, thus reducing the number of libraries required to run XiniX and as a result changes to "mainstream" libs won't have any bearing affect since they're not used. > > > Regarding the musl switch over, I had no clue that uClibc has been > > shelved for so long! > > Yeah, it got run over by buildroot as I explained here: > > http://landley.net/aboriginal/history.html > > (I _really_ need to finish that page at some point.) > > In theory uClibc development is unstuck now but its baseline development > rate is way way way slower than it used to be, because the project is > bloated and has decades of history nobody really seems to understand > anymore. > > Here's the start of a long "uClibc has a problem" thread from 2007 > talking about stagnation _then_ which wasn't fixed for another 3-4 years: > > http://lists.uclibc.org/pipermail/uclibc/2007-September/039215.html > > I started sending the maintainer a cake when it had been a year between > releases back in 2005: > > http://lists.uclibc.org/pipermail/uclibc/2005-January/010877.html > http://lists.uclibc.org/pipermail/uclibc/2006-December/037750.html > > Here's the 2006 OLS paper on NPTL support for uClibc: > > http://kernel.org/doc/ols/2006/ols2006v1-pages-409-420.pdf > > It took ~5 years for that to actually get merged and supported on all > targets. What happened was sjhill refused to merge his work until he was > paid for it, and other people got into the habit of doing extensive work > in their own private forks. The maintainer had mostly wandered off to do > other things (buildroot, his own business) and didn't police this, so it > got to the point where they were actively driving _away_ anybody who > committed to the main fork (because it made them do extra work merging > stuff into their private forks). > > Towards the end of 2008 I appointed a new uClibc maintainer more or less > by fiat to try to fix this: > > http://lists.uclibc.org/pipermail/uclibc/2008-October/041191.html > > But they didn't get NPTL support into all architectures and actually > working for something like two years after that. Basically the tree was > such a horrible mess at that point, with at least _three_ independent > NPTL implementations for various architectures (because sjhill wouldn't > show anybody his code), that cleanup work took a long time. > > Development finally got moving forward again under that new maintainer a > couple years ago, but it's very slow and there's a huge amount of > inertia. The eglibc project arose because of the vacuum created by > uClibc's stagnation, for example. > > The upshot is that uClibc is no longer _simple_, and the "first mostly > works" version (0.9.24) was in 2003, and was the _eighth_ release that > year. There were then 2 releases in 2004, one in 2005, and NONE in the > whole of 2006. (Hence the sending of cakes.) The project still hasn't > recovered from that. > Wow what a mess! > > > Why wouldn't the switch to musl be more critical? > > musl currently supports x86, x86-64, and arm. I currently have mips, > sh4, sparc, and powerpc working under uClibc. > > There's a lot more potential in musl, but it's not fleshed out yet. > How often is your software used with those other architectures? Obviously the most used are already covered under musl and with continued development with it instead of the stagnant uClibc, you could expand your coverage of the desired architectures in probably the same amount of time! :) > > > Not only would you get actively developed software, but the speed > > increases as shown in one of the links below. > > I'm not doing it to speed up code, I'm doing it so that if I post a bug > report to the mailing list (and follow up repeatedly) it won't get > ignored for a year and a half. > > I'd also like to switch to a libc implementation that hasn't got THREE > separate threading implementations. > > http://lists.busybox.net/pipermail/uclibc/2008-September/041023.html > > And one where x86-64 support isn't a buggy afterthought: > > http://lists.uclibc.org/pipermail/uclibc/2011-October/045835.html > > (And no, glibc isn't even in consideration. It requires perl to build.) > Speed may not be a primary reason, but it sure is a nice benefit of a switch! :) Your points above would seem to make my point even more valid about the switch from uClibc to musl. > > > I can see your point about standardization in the qwerty keyboard > > example provided, however, because of how Linux has evolved compared to > > the design goals of the XiniX distro, it's easier to manage the OS by > > separating it as I have. Otherwise, there would be an unwieldy amount > > of mounts to maintain with software and configs going in various places > > (e.g. /bin, /usr/bin, /usr/local/bin, /usr/share, etc). The layout for > > XiniX is uniform and makes sense where things are located and why. > > Good luck reinventing the wheel. > Thanks, hopefully this one will be made of rubber and not stone! :) > > > Thanks for all the articles on the GPLv2 vs GPLv3 - lots of reading to > > do! I'm sure I'll have more questions for you once I'm done. :) > > I need to write up a unified explanation of that. I need to finish the > history page I linked to earlier. I need to write up the > git-bisect-howto follow up I've had half-finished for 3 months. I need > to catch up on my email... > > Rob > Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob at landley.net Mon Aug 13 20:52:06 2012 From: rob at landley.net (Rob Landley) Date: Mon, 13 Aug 2012 15:52:06 -0500 Subject: [Aboriginal] greetings! In-Reply-To: References: <4FFB9E95.9050302@landley.net> <4FFDACF5.9050108@landley.net> Message-ID: <502968F6.7060306@landley.net> On 07/12/2012 09:24 AM, David Henderson wrote: > Wow and I often complain about my schedule! :) Seems like you're > working on 28 hour work days! lol This might be a good opportunity for > you to possibly offload some work then - at least on the distro > creation/maintenance side. Apparently, a chunk of messages from a month ago got missed. Catching up... (It's possible I already answered this one, but if so I don't remember it.) > My replies are in-line below to prevent confusion... ... > Define "their work"? > > I did "boot to a shell prompt on serial console, with native development > tools", targeting various qemu platforms. This let people build code > natively. > > I didn't really tackle "what tools do you need on target", because > everybody's needs are different and if you can build it natively, you've > got wget and a toolchain. > > The proliferation of boards is kind of amazing. A half-dozen projects > from buildroot to OpenEmbedded accumulated buckets of board > configurations, but never cleanly separated them out from toolchain and > root filesystem config, and I decided the correct thing to do was get > defconfigs upstream into linus's git. > > Their "work" would be anything the user normally does during the day: > email, word processing, listen to music, watch videos, program, etc. This is not a bounded set. Different people do different things. > While the distro won't contain all of the software for everyones "work" > (e.g. CAD users), it'll start with a set of software to perform the most > common tasks. Unfortunately the "most common tasks" really isn't a bounded set either. People are strange. > As time elapses, more and more can be added to include > users with more rare "work". This software, however, will be installed > with their "personal data" as mentioned below. Good luck with it. I think that's a can of worms. > > 2. Although the OS can be installed and run from a storage device, > it's > > intended to take a minimal amount of memory as it's home - RAM based. > > This should make upgrading as simple as replacing a single file! > > I remember ten years ago working out how to combine a kernel and root > filesystem into a single file, and patching lilo to have a "length" > parameter so it would only load the kernel and not the whole root > filesystem into memory. > > http://landley.net/aboriginal/old > > I even worked out some unused space in the ELF header (for user mode > linux, the emulator I was using at the time) which was the old floppy > boot sector on real kernels so either way "unused space" that I could > stamp a 4 byte length to say what offset losetup should use creating the > loopback file for the root filesystem. (I used zisofs because squashfs > wasn't merged yet.) Getting an initrd built into the kernel so it could > do the loopback setup and hand off is what got me poking at initramfs in > the first place... > > Good times, good times... > > > Wow thats pretty interesting! Currently XiniX uses grub, which I don't > think has an option like that. Neither did lilo, hence http://landley.net/aboriginal/old/lilo-length.patch > Also, XiniX has the "firmware" and > Kernel separated (I didn't even know those two could be combined). Firmware turns out not to be a well-defined term, but between initramfs and Greg Ungerer's "Linux without a bootloader" (slides http://elinux.org/ELC_2010_Presentations video at http://free-electrons.com/blog/elce-2010-videos/ ) there's no reason you _can't_ do such a thing today. But it would be fairly hardware-specific. > I suppose the integration of the two can help maintain system stability > since everything is tested and released together... Not necessarily. Having orthogonal modules you can test independently and swap out different implementations of can help make the system work better too. Otherwise you get horrible version skew instability like the way udev only ever works with specific kernel versions. > Did you ever run > into any negative aspects when working like this? The only one I can > see off the top of my head is that if the distro isn't updated > routinely, the kernel will get dated and present exploitable bugs to > crackers. If you're allowing arbitrary third parties to run random userspace code on your system, it needs to be administered. But if all it's got is a simple web server serving static pages out of a chroot there may not actually _be_ any way to exploit it. But the packaging mechanism was only mildly interesting, and I stopped doing it years ago because the bootloader built into qemu was a better solution for what I was trying to do. > > 3. Having an easily usable interface that's common among the various > > devices while being as simplistic and efficient as possible. > > Shell prompt? xfce? Web gui for headless boxes? > > Just getting the _drivers_ right for a wide variety of boards was a huge > pain for me, and I was focusing on qemu. And I have to regression test > them every kernel release and bisect some really nasty stuff sometimes. > (I'm doing a writeup the most recent round of git bisect to teach people > how to bisect bugs under adverse conditions. The one that broke armv5l's > scsi driver was buried several bugs deep behind "the serial console > broke", "build break", and so on...) > > Currently XiniX is CLI only, but hopefully by the end of next month the > graphical side will be up and running. So it's been a month. How did that go? > In order to keep the size down > while maintaining flexibility, the GUI is entirely web-based so it works > with head and headless devices in a uniform, consistent way. Also it > does have software such as QT and/or GTK installed for software that > currently doesn't have a web-interface-only alternative (and is also > required for the webkit backend). Back up: what problem are you trying to solve? With Aboriginal Linux, I'm solving the problem of cross compiling, by trying to eliminate anyone else's need to do it (at least as part of Linux development). The solution is in two parts: substitute native Linux development environments running under emulation, and perform the cross compiling necessary to _get_ to that native development environment in a reproducible way for a lot of different plaforms (so you're not trusting my magic binaries but don't have to do it yourself either). In the process I'm answering some related questions: "how small and simple can I get a native development environment" (to reduce the bits that need to be cross compiled), how do I get the utterly sucky performance of emulators to a point people are willing to put up with it (pick a good emulator and install distcc to help out), and "how do you bootstrap up from there to more complex native build environments in a target-agnostic way" (all those build-images as examples). But the core idea here is "get rid of the need for cross compiling", and that lets me know where my project STOPS. It lets me know I don't care about drivers for specific boards, that I don't care about QT, that I don't care about package management on the target, or target code installation (bootloaders, jtags, sdcard partitioning)... These are all things where once people have a working native development environment, they can take it from there. I don't _have_ to solve that for them. Your project seems to have a less clear dividing line between what you do and what you DON'T do. If you can't say "this is not part of this project", you get feature creep... > Maintaining stability for various aspects such as the drivers mentioned > above can be a pain. While there will have to be testing and such for > third party hardware, the business will release "official" hardware that > will obviously be tested with the XiniX distro so that these issues > won't be a factor as often. Do you have any idea how _much_ third party hardware there is out there? > > 4. A better, more logical division of the system and user for various > > reasons. > > Um, like Apple did with macosx? Like that /usr->/usr/bin move? Like > gobolinux where they just went "screw it" and moved everything for no > apparent reason? > > > Yeah something like that. :) For example XiniX actually doesn't have a > /usr directory at all, but instead software is considered "system" > (which will be added to the firmware /bin, /sbin), "personal" (~/.bin, > ~/.sbin), or "community" (/home/public/.bin, /home/public/.sbin). This > is done, again, to separate the system from the user/userbase and is > necessary for some of the design goals. There are a few other > non-traditional aspects of the system too. Apple got away with it because they're apple. Fedora is trying to make a very minimal change in a backwards compatible way. You're adding _hidden_ executable directories... I really don't see that ending well. > > While those are just a handful of ideas behind the distro, I'm not > sure > > if that's what you were looking for. Currently the distro (named > XiniX > > - pronounced "zen ics") uses busybox, but I'd love to replace it with > > toybox once the required binaries are incorporated. > > I'm all for it... > > > Great! I'll keep my eye on the development and incorporate it as soon > as I can! I'm working on it. Got a bit mugged by real life again... > That's pretty much where I stop. I provide a basic tool, what people do > with it from there I'm intentionally agnostic about because there are a > more use cases for a knife than for nail clippers and vegetable peelers > combined. > > That simple system would be ideal for a VM server though, don't you > think? After all, that system should *not* contain anything but the > necessities to boot and run whatever, more complex systems via VM's right? Sure. I'm always happy when people find a use for the tools I provide, but I'm trying hard to stay agnostic about what use those tools might be put to. I'll happily respond to "I need it to do X because Y", that's great feedback. But "my hammer keeps scratching the surfaces I'm pounding dents out of" means I say "here's how you can wrap some cloth around it, and some example cloth", not "from now on it will come wrapped in cloth by default". > So are you trying to accomplish the goal of developing an SDK with > aboriginal Linux? What are your goals and intended audience with that > distro? See above. "Make cross compiling go away." Note that cross compiling includes "building a uClibc system under a glibc system", and next up a musl system. (Assembling a toolchain in general is hard and fiddly.) > Rob > -- > GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. > Either it's "mere aggregation", or a license violation. Pick one. > > > Looking at your signature, it reminded me of another questions I wanted > to ask you. If I were to guess, I'd say that you were more of a GPLv2 > guy. Having said that, and I know this could take up pages worth of > information, but as an overview why the dislike for the GPLv3? Perhaps > 'dislike' isn't the right word, but I'm sure you understand what I'm asking. I need to finish writing up my thing on that. I've got a long file full of notes... > As always, thanks for your time! > Dave Sorry I'm so far behind. I need to amend the kernel MAINTAINERS file to exclude the device trees subdirectory, but I've been holding off until I get the PGP signatures to reestablish kernel.org access so I can put a git tree up there I can submit stuff through, and sometimes "I refuse to fix this until I do it RIGHT" can lead to piles of side effects imposed upon oneself as a weird sort of peanance. My inbox is like that right now. Rob -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1344891126.0 From dhenderson at digital-pipe.com Tue Aug 14 13:02:08 2012 From: dhenderson at digital-pipe.com (David Henderson) Date: Tue, 14 Aug 2012 09:02:08 -0400 Subject: [Aboriginal] greetings! In-Reply-To: <502968F6.7060306@landley.net> References: <4FFB9E95.9050302@landley.net> <4FFDACF5.9050108@landley.net> <502968F6.7060306@landley.net> Message-ID: Nice to hear from you Rob! The below thread was already part of the discussion, but it was good to see additional input. :) The latest mail should be dated Aug 2. Dave On Mon, Aug 13, 2012 at 4:52 PM, Rob Landley wrote: > On 07/12/2012 09:24 AM, David Henderson wrote: > > Wow and I often complain about my schedule! :) Seems like you're > > working on 28 hour work days! lol This might be a good opportunity for > > you to possibly offload some work then - at least on the distro > > creation/maintenance side. > > Apparently, a chunk of messages from a month ago got missed. Catching up... > > (It's possible I already answered this one, but if so I don't remember it.) > > > My replies are in-line below to prevent confusion... > ... > > Define "their work"? > > > > I did "boot to a shell prompt on serial console, with native > development > > tools", targeting various qemu platforms. This let people build code > > natively. > > > > I didn't really tackle "what tools do you need on target", because > > everybody's needs are different and if you can build it natively, > you've > > got wget and a toolchain. > > > > The proliferation of boards is kind of amazing. A half-dozen projects > > from buildroot to OpenEmbedded accumulated buckets of board > > configurations, but never cleanly separated them out from toolchain > and > > root filesystem config, and I decided the correct thing to do was get > > defconfigs upstream into linus's git. > > > > Their "work" would be anything the user normally does during the day: > > email, word processing, listen to music, watch videos, program, etc. > > This is not a bounded set. Different people do different things. > > > While the distro won't contain all of the software for everyones "work" > > (e.g. CAD users), it'll start with a set of software to perform the most > > common tasks. > > Unfortunately the "most common tasks" really isn't a bounded set either. > People are strange. > > > As time elapses, more and more can be added to include > > users with more rare "work". This software, however, will be installed > > with their "personal data" as mentioned below. > > Good luck with it. I think that's a can of worms. > > > > 2. Although the OS can be installed and run from a storage device, > > it's > > > intended to take a minimal amount of memory as it's home - RAM > based. > > > This should make upgrading as simple as replacing a single file! > > > > I remember ten years ago working out how to combine a kernel and root > > filesystem into a single file, and patching lilo to have a "length" > > parameter so it would only load the kernel and not the whole root > > filesystem into memory. > > > > http://landley.net/aboriginal/old > > > > I even worked out some unused space in the ELF header (for user mode > > linux, the emulator I was using at the time) which was the old floppy > > boot sector on real kernels so either way "unused space" that I could > > stamp a 4 byte length to say what offset losetup should use creating > the > > loopback file for the root filesystem. (I used zisofs because > squashfs > > wasn't merged yet.) Getting an initrd built into the kernel so it > could > > do the loopback setup and hand off is what got me poking at > initramfs in > > the first place... > > > > Good times, good times... > > > > > > Wow thats pretty interesting! Currently XiniX uses grub, which I don't > > think has an option like that. > > Neither did lilo, hence > http://landley.net/aboriginal/old/lilo-length.patch > > > Also, XiniX has the "firmware" and > > Kernel separated (I didn't even know those two could be combined). > > Firmware turns out not to be a well-defined term, but between initramfs > and Greg Ungerer's "Linux without a bootloader" (slides > http://elinux.org/ELC_2010_Presentations video at > http://free-electrons.com/blog/elce-2010-videos/ ) there's no reason you > _can't_ do such a thing today. But it would be fairly hardware-specific. > > > I suppose the integration of the two can help maintain system stability > > since everything is tested and released together... > > Not necessarily. Having orthogonal modules you can test independently > and swap out different implementations of can help make the system work > better too. Otherwise you get horrible version skew instability like the > way udev only ever works with specific kernel versions. > > > Did you ever run > > into any negative aspects when working like this? The only one I can > > see off the top of my head is that if the distro isn't updated > > routinely, the kernel will get dated and present exploitable bugs to > > crackers. > > If you're allowing arbitrary third parties to run random userspace code > on your system, it needs to be administered. But if all it's got is a > simple web server serving static pages out of a chroot there may not > actually _be_ any way to exploit it. > > But the packaging mechanism was only mildly interesting, and I stopped > doing it years ago because the bootloader built into qemu was a better > solution for what I was trying to do. > > > > 3. Having an easily usable interface that's common among the > various > > > devices while being as simplistic and efficient as possible. > > > > Shell prompt? xfce? Web gui for headless boxes? > > > > Just getting the _drivers_ right for a wide variety of boards was a > huge > > pain for me, and I was focusing on qemu. And I have to regression > test > > them every kernel release and bisect some really nasty stuff > sometimes. > > (I'm doing a writeup the most recent round of git bisect to teach > people > > how to bisect bugs under adverse conditions. The one that broke > armv5l's > > scsi driver was buried several bugs deep behind "the serial console > > broke", "build break", and so on...) > > > > Currently XiniX is CLI only, but hopefully by the end of next month the > > graphical side will be up and running. > > So it's been a month. How did that go? > > > In order to keep the size down > > while maintaining flexibility, the GUI is entirely web-based so it works > > with head and headless devices in a uniform, consistent way. Also it > > does have software such as QT and/or GTK installed for software that > > currently doesn't have a web-interface-only alternative (and is also > > required for the webkit backend). > > Back up: what problem are you trying to solve? > > With Aboriginal Linux, I'm solving the problem of cross compiling, by > trying to eliminate anyone else's need to do it (at least as part of > Linux development). > > The solution is in two parts: substitute native Linux development > environments running under emulation, and perform the cross compiling > necessary to _get_ to that native development environment in a > reproducible way for a lot of different plaforms (so you're not trusting > my magic binaries but don't have to do it yourself either). > > In the process I'm answering some related questions: "how small and > simple can I get a native development environment" (to reduce the bits > that need to be cross compiled), how do I get the utterly sucky > performance of emulators to a point people are willing to put up with it > (pick a good emulator and install distcc to help out), and "how do you > bootstrap up from there to more complex native build environments in a > target-agnostic way" (all those build-images as examples). > > But the core idea here is "get rid of the need for cross compiling", and > that lets me know where my project STOPS. > > It lets me know I don't care about drivers for specific boards, that I > don't care about QT, that I don't care about package management on the > target, or target code installation (bootloaders, jtags, sdcard > partitioning)... These are all things where once people have a working > native development environment, they can take it from there. I don't > _have_ to solve that for them. > > Your project seems to have a less clear dividing line between what you > do and what you DON'T do. If you can't say "this is not part of this > project", you get feature creep... > > > Maintaining stability for various aspects such as the drivers mentioned > > above can be a pain. While there will have to be testing and such for > > third party hardware, the business will release "official" hardware that > > will obviously be tested with the XiniX distro so that these issues > > won't be a factor as often. > > Do you have any idea how _much_ third party hardware there is out there? > > > > 4. A better, more logical division of the system and user for > various > > > reasons. > > > > Um, like Apple did with macosx? Like that /usr->/usr/bin move? Like > > gobolinux where they just went "screw it" and moved everything for no > > apparent reason? > > > > > > Yeah something like that. :) For example XiniX actually doesn't have a > > /usr directory at all, but instead software is considered "system" > > (which will be added to the firmware /bin, /sbin), "personal" (~/.bin, > > ~/.sbin), or "community" (/home/public/.bin, /home/public/.sbin). This > > is done, again, to separate the system from the user/userbase and is > > necessary for some of the design goals. There are a few other > > non-traditional aspects of the system too. > > Apple got away with it because they're apple. Fedora is trying to make a > very minimal change in a backwards compatible way. You're adding > _hidden_ executable directories... I really don't see that ending well. > > > > While those are just a handful of ideas behind the distro, I'm not > > sure > > > if that's what you were looking for. Currently the distro (named > > XiniX > > > - pronounced "zen ics") uses busybox, but I'd love to replace it > with > > > toybox once the required binaries are incorporated. > > > > I'm all for it... > > > > > > Great! I'll keep my eye on the development and incorporate it as soon > > as I can! > > I'm working on it. Got a bit mugged by real life again... > > > That's pretty much where I stop. I provide a basic tool, what people > do > > with it from there I'm intentionally agnostic about because there > are a > > more use cases for a knife than for nail clippers and vegetable > peelers > > combined. > > > > That simple system would be ideal for a VM server though, don't you > > think? After all, that system should *not* contain anything but the > > necessities to boot and run whatever, more complex systems via VM's > right? > > Sure. I'm always happy when people find a use for the tools I provide, > but I'm trying hard to stay agnostic about what use those tools might be > put to. > > I'll happily respond to "I need it to do X because Y", that's great > feedback. But "my hammer keeps scratching the surfaces I'm pounding > dents out of" means I say "here's how you can wrap some cloth around it, > and some example cloth", not "from now on it will come wrapped in cloth > by default". > > > So are you trying to accomplish the goal of developing an SDK with > > aboriginal Linux? What are your goals and intended audience with that > > distro? > > See above. "Make cross compiling go away." > > Note that cross compiling includes "building a uClibc system under a > glibc system", and next up a musl system. (Assembling a toolchain in > general is hard and fiddly.) > > > Rob > > -- > > GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. > > Either it's "mere aggregation", or a license violation. Pick one. > > > > > > Looking at your signature, it reminded me of another questions I wanted > > to ask you. If I were to guess, I'd say that you were more of a GPLv2 > > guy. Having said that, and I know this could take up pages worth of > > information, but as an overview why the dislike for the GPLv3? Perhaps > > 'dislike' isn't the right word, but I'm sure you understand what I'm > asking. > > I need to finish writing up my thing on that. I've got a long file full > of notes... > > > As always, thanks for your time! > > Dave > > Sorry I'm so far behind. I need to amend the kernel MAINTAINERS file to > exclude the device trees subdirectory, but I've been holding off until I > get the PGP signatures to reestablish kernel.org access so I can put a > git tree up there I can submit stuff through, and sometimes "I refuse to > fix this until I do it RIGHT" can lead to piles of side effects imposed > upon oneself as a weird sort of peanance. My inbox is like that right now. > > Rob > -- > GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. > Either it's "mere aggregation", or a license violation. Pick one. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob at landley.net Thu Aug 16 11:20:56 2012 From: rob at landley.net (Rob Landley) Date: Thu, 16 Aug 2012 06:20:56 -0500 Subject: [Aboriginal] Fwd: greetings! In-Reply-To: References: <4FFB9E95.9050302@landley.net> <4FFDACF5.9050108@landley.net> <50076C48.5010403@landley.net> <501044B6.5010504@landley.net> <50190F26.5000405@landley.net> Message-ID: <502CD798.7070109@landley.net> My inbox got out of control and I'm catching up, but about half a month behind. If I answer some duplicates, now you know why. On 08/02/2012 09:52 AM, David Henderson wrote: > On Wed, Aug 1, 2012 at 7:12 AM, Rob Landley > wrote: > > On 07/26/2012 08:17 AM, David Henderson wrote: > > This email thread has started to become soup! So... I'll start a new > > paragraph so we can start over. :) > > > > Going back to the software included in the "firmware", the list should > > be very small and only allow the device to get to the point for user > > interaction. For example, > > > > - busybox (later toybox) for core functionality including user > > management, scheduling, etc > > - bash > > - clAPI (our own software) > > - the appropriate mount binaries for ext, ntfs, nfs, and smb/cifs > > Once upon a time I wrote the busybox mount. (Well, rewrote it 3 times > until there was nothing left of the original.) > > smb and nfs shouldn't need mount binaries, you can mount them with the > right -o command line. All the mount binary would do is prompt you for > password so you don't have to say pass= in the -o line in plaintext. > > ext? shouldn't need a special mount binary, just the standard one and > the kernel module (if it's not static). > > ntfs I'm not familiar with. > > I did a quick look at the busybox mount help, but didn't see any -o > parameters for specifying certain values (e.g. user,pass,dom,etc) to > mount network shares. Perhaps some testing is in order! :) it worked fine circa 2006, which is when I stopped banging on it. user= and pass= are kernel level options. Any unrecognized options are (or at least should be) passed to the kernel as a comma-separated string for parsing by the filesystem driver. Only the VFS flag options should get spliced out, and I've been re-examining those doing research for toybox mount. http://lists.landley.net/pipermail/toybox-landley.net/2012-August/000628.html The problem with pass is you don't want to say "pass=" on the command line or other users on the same system could see it, which is why you have a helper prompt you for the password from stdin. But if you don't mind that, supplying it from the command line should work. Not sure the escape syntax for commas, might need to be the last option in the list. The other problem is the kernel can't (sanely) do DNS lookup, so you need to resolve names to IPs after recognizing such in the mount URL format, which is the other thing a mount helper does. > The XiniX > "firmware" already has several external mount binaries included > (mount.cifs and mount.ntfs), so it shouldn't be a problem either way. :) Good luck with it then. > > - ssh > > I've got static dropbear binaries up at > http://landley.net/aboriginal/bin natively built by a control-image. > > Have you run into any problems with dropbear over openSSH? Aside from > the small size, are there any benefits of using dropbear? Not really. The big frustration is that dropbear doesn't supply ssl. I've poked at adding stunnel but it's one of my 8 gazillion "if I had a spare summer" things... > > - ssl (for secured headless devices or other communication) > > How do you get anything else to use it? Dropbear is self-contained the > busybox web server and wget don't link against eternal libraries. (I > thought about makign them use stunnel but never got around to it. Did > that happen in my absence?) > > Shouldn't this be installed for (optional) secured headless web > interfaces? As an alternative to wget, curl can be used (which can be > used with openSSL). Any advantage either way? In theory wget could be used with stunnel. I generally find curl to be overcomplicted, but most people don't care and just use what's provided. Neither is in SUSv4 or LSB. > > - ssmtp (or similar) for automated outbound communication of device > > busybox has had a "sendmail" command for a while now. > > How easy is it to get the BB sendmail working? I was able to get sSMTP > up and running in a few minutes... Not a clue, never used it. Just noting it's there. > > * optional alsa (for sound enabled devices) > > * optional graphics software for headed devices (headless will already > > be handled via clAPI) > > Those were always very device-specific to me, but I've never understood > the various incarnations of the linux sound layer all that well (just > enough to really hate pulseaudio), and the phrase "graphics software for > headless devices" reads like "hats for the headless horseman"... > > Yes, the "graphics software for *headed* devices" is somewhat general, > but will include things like GTK, QT, Awesome WM, etc). gtk _and_ qt. > This area is > device-specific, but can be setup easily via the installation of the > "firmware" for the device. *shrug* Good luck. > > Items such as mplayer, samba, rdesktop, okular, perl, rsync, etc > will be > > outside the scope of the "firmware" because they aren't required > to turn > > on a device, login, and get to either the prompt or blank desktop > > (GUI). The list above should enable a device to communicate/interact > > with other devices on the most basic level, but nothing beyond. > > Because the "firmware" is kept to an absolute minimum, I think any > > library API changes will not really disrupt the device from working. > > Well, good luck with it. > > I take it you don't think that will be the case. :) Let's say I'm unlikely to believe it before seeing it. > How many libraries > will be used with the basic set of software listed above? It should be > far less than a typical distro, thus reducing the number of libraries > required to run XiniX and as a result changes to "mainstream" libs won't > have any bearing affect since they're not used. I remember alpine linux, ubuntu jeos, and puppy linux... The problem is there doesn't seem to be an obvious small set here. > The upshot is that uClibc is no longer _simple_, and the "first mostly > works" version (0.9.24) was in 2003, and was the _eighth_ release that > year. There were then 2 releases in 2004, one in 2005, and NONE in the > whole of 2006. (Hence the sending of cakes.) The project still hasn't > recovered from that. > > Wow what a mess! *shrug* Bernard's doing his best, and it's nice to see it maintained again. > > Why wouldn't the switch to musl be more critical? > > musl currently supports x86, x86-64, and arm. I currently have mips, > sh4, sparc, and powerpc working under uClibc. > > There's a lot more potential in musl, but it's not fleshed out yet. > > How often is your software used with those other architectures? How often does anybody use sparc for anything? http://lists.gnu.org/archive/html/qemu-devel/2010-02/msg01238.html > Obviously the most used are already covered under musl and with > continued development with it instead of the stagnant uClibc, you could > expand your coverage of the desired architectures in probably the same > amount of time! :) They're adding more, and qemu helps. > > Not only would you get actively developed software, but the speed > > increases as shown in one of the links below. > > I'm not doing it to speed up code, I'm doing it so that if I post a bug > report to the mailing list (and follow up repeatedly) it won't get > ignored for a year and a half. > > I'd also like to switch to a libc implementation that hasn't got THREE > separate threading implementations. > > http://lists.busybox.net/pipermail/uclibc/2008-September/041023.html > > And one where x86-64 support isn't a buggy afterthought: > > http://lists.uclibc.org/pipermail/uclibc/2011-October/045835.html > > (And no, glibc isn't even in consideration. It requires perl to build.) > > > Speed may not be a primary reason, but it sure is a nice benefit of a > switch! :) Your points above would seem to make my point even more > valid about the switch from uClibc to musl. I want to replace cross compiling with native development either on target hardware or under emulation, but I also want to make cross compiling to a new target as easy as possible to do. Cross compiling busybox instead of a dozen random gnu packages is the obvious way to go there: once you're on target you can bootstrap up to LFS (which I have a script for, which desperately needs updating because it's already a couple LFS releases behind). I want the libc equivalent of busybox: simple, self contained, does its job without too much fuss, doesn't necessarily handle ever corner case in the universe. Just the "there are three complete threading implementations in this library and I don't know which ones work on which targets" part of uClibc would have me looking around... Rob -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1345116056.0 From dhenderson at digital-pipe.com Thu Aug 16 13:50:15 2012 From: dhenderson at digital-pipe.com (David Henderson) Date: Thu, 16 Aug 2012 09:50:15 -0400 Subject: [Aboriginal] Fwd: greetings! In-Reply-To: <502CD798.7070109@landley.net> References: <4FFB9E95.9050302@landley.net> <4FFDACF5.9050108@landley.net> <50076C48.5010403@landley.net> <501044B6.5010504@landley.net> <50190F26.5000405@landley.net> <502CD798.7070109@landley.net> Message-ID: No problem, I know how it is to get backed up once in a while. Perhaps you need to get rid of any non-essential responsibilities to give you some more "free time". :) On Thu, Aug 16, 2012 at 7:20 AM, Rob Landley wrote: > My inbox got out of control and I'm catching up, but about half a month > behind. If I answer some duplicates, now you know why. > > On 08/02/2012 09:52 AM, David Henderson wrote: > > On Wed, Aug 1, 2012 at 7:12 AM, Rob Landley > > wrote: > > > > On 07/26/2012 08:17 AM, David Henderson wrote: > > > This email thread has started to become soup! So... I'll start a > new > > > paragraph so we can start over. :) > > > > > > Going back to the software included in the "firmware", the list > should > > > be very small and only allow the device to get to the point for > user > > > interaction. For example, > > > > > > - busybox (later toybox) for core functionality including user > > > management, scheduling, etc > > > - bash > > > - clAPI (our own software) > > > - the appropriate mount binaries for ext, ntfs, nfs, and smb/cifs > > > > Once upon a time I wrote the busybox mount. (Well, rewrote it 3 times > > until there was nothing left of the original.) > > > > smb and nfs shouldn't need mount binaries, you can mount them with > the > > right -o command line. All the mount binary would do is prompt you > for > > password so you don't have to say pass= in the -o line in plaintext. > > > > ext? shouldn't need a special mount binary, just the standard one and > > the kernel module (if it's not static). > > > > ntfs I'm not familiar with. > > > > I did a quick look at the busybox mount help, but didn't see any -o > > parameters for specifying certain values (e.g. user,pass,dom,etc) to > > mount network shares. Perhaps some testing is in order! :) > > it worked fine circa 2006, which is when I stopped banging on it. user= > and pass= are kernel level options. Any unrecognized options are (or at > least should be) passed to the kernel as a comma-separated string for > parsing by the filesystem driver. Only the VFS flag options should get > spliced out, and I've been re-examining those doing research for toybox > mount. > > > http://lists.landley.net/pipermail/toybox-landley.net/2012-August/000628.html > > The problem with pass is you don't want to say "pass=" on the command > line or other users on the same system could see it, which is why you > have a helper prompt you for the password from stdin. But if you don't > mind that, supplying it from the command line should work. Not sure the > escape syntax for commas, might need to be the last option in the list. > > The other problem is the kernel can't (sanely) do DNS lookup, so you > need to resolve names to IPs after recognizing such in the mount URL > format, which is the other thing a mount helper does. > Nice to know! That would explain why they're missing from the BB help. :) > > > The XiniX > > "firmware" already has several external mount binaries included > > (mount.cifs and mount.ntfs), so it shouldn't be a problem either way. :) > > Good luck with it then. > > > > - ssh > > > > I've got static dropbear binaries up at > > http://landley.net/aboriginal/bin natively built by a control-image. > > > > Have you run into any problems with dropbear over openSSH? Aside from > > the small size, are there any benefits of using dropbear? > > Not really. The big frustration is that dropbear doesn't supply ssl. > I've poked at adding stunnel but it's one of my 8 gazillion "if I had a > spare summer" things... > If dropbear doesn't have security without a wrapper such as stunnel, then I'd say there really isn't any cost savings by using it. Briefly looking into stunnel informed me that it relies on OpenSSL (which would already need to be installed to get shttp working). Plus it looks like dropbear is only "ssh", not scp or sftp, providing 1/3 of the benefits. Thoughts? > > > > - ssl (for secured headless devices or other communication) > > > > How do you get anything else to use it? Dropbear is self-contained > the > > busybox web server and wget don't link against eternal libraries. (I > > thought about makign them use stunnel but never got around to it. Did > > that happen in my absence?) > > > > Shouldn't this be installed for (optional) secured headless web > > interfaces? As an alternative to wget, curl can be used (which can be > > used with openSSL). Any advantage either way? > > In theory wget could be used with stunnel. > > I generally find curl to be overcomplicted, but most people don't care > and just use what's provided. Neither is in SUSv4 or LSB. > cURL actually looks like it's more lightweight than wget too. :) > > > > - ssmtp (or similar) for automated outbound communication of device > > > > busybox has had a "sendmail" command for a while now. > > > > How easy is it to get the BB sendmail working? I was able to get sSMTP > > up and running in a few minutes... > > Not a clue, never used it. Just noting it's there. > Ah, gotcha! I'll look into it when I get to that point... > > > > * optional alsa (for sound enabled devices) > > > * optional graphics software for headed devices (headless will > already > > > be handled via clAPI) > > > > Those were always very device-specific to me, but I've never > understood > > the various incarnations of the linux sound layer all that well (just > > enough to really hate pulseaudio), and the phrase "graphics software > for > > headless devices" reads like "hats for the headless horseman"... > > > > Yes, the "graphics software for *headed* devices" is somewhat general, > > but will include things like GTK, QT, Awesome WM, etc). > > gtk _and_ qt. > GTK will have to be installed for minimal local GUI since one of the main packages relies on it. QT should just be made available as a installable package via the software repository. As a side note, what do you think of wxWidgets? It's the lightest of all the GUI toolkits, but does have some disadvantages too... > > > This area is > > device-specific, but can be setup easily via the installation of the > > "firmware" for the device. > > *shrug* Good luck. > > > > > Items such as mplayer, samba, rdesktop, okular, perl, rsync, etc > > will be > > > outside the scope of the "firmware" because they aren't required > > to turn > > > on a device, login, and get to either the prompt or blank desktop > > > (GUI). The list above should enable a device to > communicate/interact > > > with other devices on the most basic level, but nothing beyond. > > > Because the "firmware" is kept to an absolute minimum, I think any > > > library API changes will not really disrupt the device from > working. > > > > Well, good luck with it. > > > > I take it you don't think that will be the case. :) > > Let's say I'm unlikely to believe it before seeing it. > > > How many libraries > > will be used with the basic set of software listed above? It should be > > far less than a typical distro, thus reducing the number of libraries > > required to run XiniX and as a result changes to "mainstream" libs won't > > have any bearing affect since they're not used. > > I remember alpine linux, ubuntu jeos, and puppy linux... > > The problem is there doesn't seem to be an obvious small set here. > I don't think those distros have the same design goals as XiniX though... :) > > > The upshot is that uClibc is no longer _simple_, and the "first > mostly > > works" version (0.9.24) was in 2003, and was the _eighth_ release > that > > year. There were then 2 releases in 2004, one in 2005, and NONE in > the > > whole of 2006. (Hence the sending of cakes.) The project still hasn't > > recovered from that. > > > > Wow what a mess! > > *shrug* Bernard's doing his best, and it's nice to see it maintained again. > > > > Why wouldn't the switch to musl be more critical? > > > > musl currently supports x86, x86-64, and arm. I currently have mips, > > sh4, sparc, and powerpc working under uClibc. > > > > There's a lot more potential in musl, but it's not fleshed out yet. > > > > How often is your software used with those other architectures? > > How often does anybody use sparc for anything? > > http://lists.gnu.org/archive/html/qemu-devel/2010-02/msg01238.html > > > Obviously the most used are already covered under musl and with > > continued development with it instead of the stagnant uClibc, you could > > expand your coverage of the desired architectures in probably the same > > amount of time! :) > > They're adding more, and qemu helps. > > > > Not only would you get actively developed software, but the speed > > > increases as shown in one of the links below. > > > > I'm not doing it to speed up code, I'm doing it so that if I post a > bug > > report to the mailing list (and follow up repeatedly) it won't get > > ignored for a year and a half. > > > > I'd also like to switch to a libc implementation that hasn't got > THREE > > separate threading implementations. > > > > > http://lists.busybox.net/pipermail/uclibc/2008-September/041023.html > > > > And one where x86-64 support isn't a buggy afterthought: > > > > http://lists.uclibc.org/pipermail/uclibc/2011-October/045835.html > > > > (And no, glibc isn't even in consideration. It requires perl to > build.) > > > > > > Speed may not be a primary reason, but it sure is a nice benefit of a > > switch! :) Your points above would seem to make my point even more > > valid about the switch from uClibc to musl. > > I want to replace cross compiling with native development either on > target hardware or under emulation, but I also want to make cross > compiling to a new target as easy as possible to do. > > Cross compiling busybox instead of a dozen random gnu packages is the > obvious way to go there: once you're on target you can bootstrap up to > LFS (which I have a script for, which desperately needs updating because > it's already a couple LFS releases behind). > > I want the libc equivalent of busybox: simple, self contained, does its > job without too much fuss, doesn't necessarily handle ever corner case > in the universe. > > Just the "there are three complete threading implementations in this > library and I don't know which ones work on which targets" part of > uClibc would have me looking around... > > Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob at landley.net Mon Aug 20 04:47:10 2012 From: rob at landley.net (Rob Landley) Date: Sun, 19 Aug 2012 23:47:10 -0500 Subject: [Aboriginal] Fwd: mdadm issue, including qemu images. In-Reply-To: <502CB3AE.903@gmail.com> References: <4F63BFA2.3040301@gmail.com> <502CB3AE.903@gmail.com> Message-ID: <5031C14E.60305@landley.net> Ok, this is my last todo item for a release. Fedora build break: fixed. Arm versatile regression in 3.5: fixed. uClibc++ build break where it complains that .configure is a directory: fixed. On 08/16/2012 03:47 AM, Piotr Karbowski wrote: > Re-sending the images to reproduce as requested. Alright, let's look at this... Rereading our previous email exchange I see that I spent an entire evening reverse engineering your build system to figure out that the following two lines were what I needed to run to build this package the way you're doing it: wget http://www.kernel.org/pub/linux/utils/raid/mdadm/mdadm-3.2.3.tar.gz make CXFLAGS='-Os -pipe -fomit-frame-pointer' CWFLAGS='-Wall' \ mdadm.static So, under my brand new i686 dev-environment.sh I did that, and it built, and now I have an mdadm.static, and I ran it and it said --auto requires an argument. Lovely, now what... Ah, I need to add your raid image as /dev/hdc. Ok, extract the tarball you sent me, add QEMU_EXTRA="-hdc /path/to/small-raid1.qcow2" and... it says --auto requires an argument. Ok, dug up the email I'm replying to now, and that says --scan instead of --auto, and that's... still not happy. You link to a message: http://lists.uclibc.org/pipermail/uclibc/2012-March/046482.html Read, read... And there's an "mdadm.config" which you have never bothered to attach to anything, so I have to loopback mount your image to get it... except I can't, because it's an initramfs. Ok, I still have your build directory lying around here somewhere... there's no mdadm.conf in there either. Let's try grep -r... ah, it's generated by functions.sh, obviously I should have guessed. So mdadm has an --examine --scan but wants you to write the output of that to an absolute path so it can read it back in. Lovely. (Who wrote this thing?) And now it's failing because the aboriginal linux kernel doesn't have md support built in. It's in the device drivers menu, switch on just about everything in there, rebuild the kernel, boot back in... Ok: cp -r /usr/etc /tmp && mount --bind /tmp/etc /usr/etc && ./mdadm.static --examine --scan > /etc/mdadm.conf && ./mdadm.static --assemble --scan And it says: md: md1 stopped. md: hdc does not have a valid v1.0 superblock, not importing! md: md_import_device returned -22 mdadm: failed to add 22:1 to /dev/md/1: Invalid argument mdadm: /dev/md/1 assembled from 0 drives - not enough to start the array. I _think_ it's complaining about the qcow image you sent me. Possibly my host qemu isn't built with qcow support? I've now spent a couple hours trying to reproduce your test case in a dev-environment.sh boot of a stock aboriginal linux image running under qemu, and I still don't have anything that looks like the failure in your message. All of this work is still to REPRODUCE YOUR TEST CASE. If you're wondering why it takes so long to test your bug reports... now you know. You do not give me isolated orthogonal test cases demonstrating an issue in a way I can easily reproduce in an environment I'm familiar with. The majority of the work here hasn't been anything to do with aboriginal linux, it's been laboriously trying to unwrap your test case to reverse engineer your custom system to see what you're doing. Could you please get me a test case that shows my stuff failing? Pretty please? When I've built a fresh system-image-i686: whare are the minimal set of steps I do in that to reproduce your issue? Sigh. I'll try to find some time to continue hacking away at this tomorrow, but I might just cut a release and punt this to next time. > -- Piotr. Rob > -------- Original Message -------- > Subject: mdadm issue, including qemu images. > Date: Fri, 16 Mar 2012 23:33:06 +0100 > From: Piotr Karbowski > To: rob at landley.net > > Hi, we spoke on irc earlier. The quick summary is on > http://lists.uclibc.org/pipermail/uclibc/2012-March/046482.html > > if I build mdadm inside uclibc's buildroot (which use 0.9.32 uclibc) it > does work, however if I build it inside aboriginal (tested 1.1.1 and > 1.0.3) it fail on 'mdadm --assemble --auto'. Maybe kernel headers? I > have no idea. > > Attached two qemu images, one contain bootloader (extlinux), initramfs > (better-initramfs v0.5 + your strace binary) and kernel and another is > empty raid1 array with one device set as missing, without any filesystem > on it, you can run it via: > > qemu-kvm -hda better-initramfs-boot.qcow2 -hdb small-raid1.qcow2 > > If you wish to test it more, you can grab better-initramfs from > github.com/slashbeast/better-initramfs, it should be failry easy to use, > before 'make image' you can place whatever you want in sourceroot/ dir, > then replace output/initramfs.cpio.gz with the one inside > better-initramfs-boot.qcow2 and you should be good. > > > Thanks for looking into it. > > -- Piotr. > > > -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1345438030.0 From rob at landley.net Mon Aug 20 14:01:04 2012 From: rob at landley.net (Rob Landley) Date: Mon, 20 Aug 2012 09:01:04 -0500 Subject: [Aboriginal] Fwd: greetings! In-Reply-To: References: <4FFB9E95.9050302@landley.net> <4FFDACF5.9050108@landley.net> <50076C48.5010403@landley.net> <501044B6.5010504@landley.net> <50190F26.5000405@landley.net> <502CD798.7070109@landley.net> Message-ID: <50324320.50405@landley.net> A week or so back somebody suggested I drop toybox to focus on qcc. But that's not what I want to do... Rob On 08/16/2012 08:50 AM, David Henderson wrote: > No problem, I know how it is to get backed up once in a while. Perhaps > you need to get rid of any non-essential responsibilities to give you > some more "free time". :) > > > On Thu, Aug 16, 2012 at 7:20 AM, Rob Landley > wrote: > > My inbox got out of control and I'm catching up, but about half a month > behind. If I answer some duplicates, now you know why. > > On 08/02/2012 09:52 AM, David Henderson wrote: > > On Wed, Aug 1, 2012 at 7:12 AM, Rob Landley > > >> wrote: > > > > On 07/26/2012 08:17 AM, David Henderson wrote: > > > This email thread has started to become soup! So... I'll > start a new > > > paragraph so we can start over. :) > > > > > > Going back to the software included in the "firmware", the > list should > > > be very small and only allow the device to get to the point > for user > > > interaction. For example, > > > > > > - busybox (later toybox) for core functionality including user > > > management, scheduling, etc > > > - bash > > > - clAPI (our own software) > > > - the appropriate mount binaries for ext, ntfs, nfs, and > smb/cifs > > > > Once upon a time I wrote the busybox mount. (Well, rewrote it > 3 times > > until there was nothing left of the original.) > > > > smb and nfs shouldn't need mount binaries, you can mount them > with the > > right -o command line. All the mount binary would do is prompt > you for > > password so you don't have to say pass= in the -o line in > plaintext. > > > > ext? shouldn't need a special mount binary, just the standard > one and > > the kernel module (if it's not static). > > > > ntfs I'm not familiar with. > > > > I did a quick look at the busybox mount help, but didn't see any -o > > parameters for specifying certain values (e.g. user,pass,dom,etc) to > > mount network shares. Perhaps some testing is in order! :) > > it worked fine circa 2006, which is when I stopped banging on it. user= > and pass= are kernel level options. Any unrecognized options are (or at > least should be) passed to the kernel as a comma-separated string for > parsing by the filesystem driver. Only the VFS flag options should get > spliced out, and I've been re-examining those doing research for toybox > mount. > > http://lists.landley.net/pipermail/toybox-landley.net/2012-August/000628.html > > The problem with pass is you don't want to say "pass=" on the command > line or other users on the same system could see it, which is why you > have a helper prompt you for the password from stdin. But if you don't > mind that, supplying it from the command line should work. Not sure the > escape syntax for commas, might need to be the last option in the list. > > The other problem is the kernel can't (sanely) do DNS lookup, so you > need to resolve names to IPs after recognizing such in the mount URL > format, which is the other thing a mount helper does. > > > Nice to know! That would explain why they're missing from the BB help. :) > > > > > The XiniX > > "firmware" already has several external mount binaries included > > (mount.cifs and mount.ntfs), so it shouldn't be a problem either > way. :) > > Good luck with it then. > > > > - ssh > > > > I've got static dropbear binaries up at > > http://landley.net/aboriginal/bin natively built by a > control-image. > > > > Have you run into any problems with dropbear over openSSH? Aside from > > the small size, are there any benefits of using dropbear? > > Not really. The big frustration is that dropbear doesn't supply ssl. > I've poked at adding stunnel but it's one of my 8 gazillion "if I had a > spare summer" things... > > > If dropbear doesn't have security without a wrapper such as stunnel, > then I'd say there really isn't any cost savings by using it. Briefly > looking into stunnel informed me that it relies on OpenSSL (which would > already need to be installed to get shttp working). Plus it looks like > dropbear is only "ssh", not scp or sftp, providing 1/3 of the benefits. > Thoughts? > > > > > > - ssl (for secured headless devices or other communication) > > > > How do you get anything else to use it? Dropbear is > self-contained the > > busybox web server and wget don't link against eternal > libraries. (I > > thought about makign them use stunnel but never got around to > it. Did > > that happen in my absence?) > > > > Shouldn't this be installed for (optional) secured headless web > > interfaces? As an alternative to wget, curl can be used (which can be > > used with openSSL). Any advantage either way? > > In theory wget could be used with stunnel. > > I generally find curl to be overcomplicted, but most people don't care > and just use what's provided. Neither is in SUSv4 or LSB. > > > cURL actually looks like it's more lightweight than wget too. :) > > > > > > - ssmtp (or similar) for automated outbound communication of > device > > > > busybox has had a "sendmail" command for a while now. > > > > How easy is it to get the BB sendmail working? I was able to get > sSMTP > > up and running in a few minutes... > > Not a clue, never used it. Just noting it's there. > > > Ah, gotcha! I'll look into it when I get to that point... > > > > > > * optional alsa (for sound enabled devices) > > > * optional graphics software for headed devices (headless > will already > > > be handled via clAPI) > > > > Those were always very device-specific to me, but I've never > understood > > the various incarnations of the linux sound layer all that > well (just > > enough to really hate pulseaudio), and the phrase "graphics > software for > > headless devices" reads like "hats for the headless horseman"... > > > > Yes, the "graphics software for *headed* devices" is somewhat general, > > but will include things like GTK, QT, Awesome WM, etc). > > gtk _and_ qt. > > > GTK will have to be installed for minimal local GUI since one of the > main packages relies on it. QT should just be made available as a > installable package via the software repository. As a side note, what > do you think of wxWidgets? It's the lightest of all the GUI toolkits, > but does have some disadvantages too... > > > > > This area is > > device-specific, but can be setup easily via the installation of the > > "firmware" for the device. > > *shrug* Good luck. > > > > > Items such as mplayer, samba, rdesktop, okular, perl, rsync, etc > > will be > > > outside the scope of the "firmware" because they aren't required > > to turn > > > on a device, login, and get to either the prompt or blank > desktop > > > (GUI). The list above should enable a device to > communicate/interact > > > with other devices on the most basic level, but nothing beyond. > > > Because the "firmware" is kept to an absolute minimum, I > think any > > > library API changes will not really disrupt the device from > working. > > > > Well, good luck with it. > > > > I take it you don't think that will be the case. :) > > Let's say I'm unlikely to believe it before seeing it. > > > How many libraries > > will be used with the basic set of software listed above? It > should be > > far less than a typical distro, thus reducing the number of libraries > > required to run XiniX and as a result changes to "mainstream" libs > won't > > have any bearing affect since they're not used. > > I remember alpine linux, ubuntu jeos, and puppy linux... > > The problem is there doesn't seem to be an obvious small set here. > > > I don't think those distros have the same design goals as XiniX though... :) > > > > > The upshot is that uClibc is no longer _simple_, and the > "first mostly > > works" version (0.9.24) was in 2003, and was the _eighth_ > release that > > year. There were then 2 releases in 2004, one in 2005, and > NONE in the > > whole of 2006. (Hence the sending of cakes.) The project still > hasn't > > recovered from that. > > > > Wow what a mess! > > *shrug* Bernard's doing his best, and it's nice to see it maintained > again. > > > > Why wouldn't the switch to musl be more critical? > > > > musl currently supports x86, x86-64, and arm. I currently have > mips, > > sh4, sparc, and powerpc working under uClibc. > > > > There's a lot more potential in musl, but it's not fleshed out > yet. > > > > How often is your software used with those other architectures? > > How often does anybody use sparc for anything? > > http://lists.gnu.org/archive/html/qemu-devel/2010-02/msg01238.html > > > Obviously the most used are already covered under musl and with > > continued development with it instead of the stagnant uClibc, you > could > > expand your coverage of the desired architectures in probably the same > > amount of time! :) > > They're adding more, and qemu helps. > > > > Not only would you get actively developed software, but the > speed > > > increases as shown in one of the links below. > > > > I'm not doing it to speed up code, I'm doing it so that if I > post a bug > > report to the mailing list (and follow up repeatedly) it won't get > > ignored for a year and a half. > > > > I'd also like to switch to a libc implementation that hasn't > got THREE > > separate threading implementations. > > > > > http://lists.busybox.net/pipermail/uclibc/2008-September/041023.html > > > > And one where x86-64 support isn't a buggy afterthought: > > > > > http://lists.uclibc.org/pipermail/uclibc/2011-October/045835.html > > > > (And no, glibc isn't even in consideration. It requires perl > to build.) > > > > > > Speed may not be a primary reason, but it sure is a nice benefit of a > > switch! :) Your points above would seem to make my point even more > > valid about the switch from uClibc to musl. > > I want to replace cross compiling with native development either on > target hardware or under emulation, but I also want to make cross > compiling to a new target as easy as possible to do. > > Cross compiling busybox instead of a dozen random gnu packages is the > obvious way to go there: once you're on target you can bootstrap up to > LFS (which I have a script for, which desperately needs updating because > it's already a couple LFS releases behind). > > I want the libc equivalent of busybox: simple, self contained, does its > job without too much fuss, doesn't necessarily handle ever corner case > in the universe. > > Just the "there are three complete threading implementations in this > library and I don't know which ones work on which targets" part of > uClibc would have me looking around... > > > Dave -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1345471264.0 From rob at landley.net Fri Aug 24 10:45:33 2012 From: rob at landley.net (Rob Landley) Date: Fri, 24 Aug 2012 05:45:33 -0500 Subject: [Aboriginal] 1.2.0 is out! Message-ID: <50375B4D.6080501@landley.net> That took a very long time. Sorry, my day job has reduced my hobbyist programming time to somewhere around 10 hours a week if I push, by a combination of soaking up all my time and leaving me exhausted at the end of it. Of that, toybox has sucked away most of it, and being made kernel Documentation maintainer is fighting for the scraps. That said, I finally caught up with the kernel releases, and the lfs-bootstrap images for various targets are uploading now. I'm writing up a not particularly epic bisection battle I had as an example of real-world problem bisection under adverse conditinos. The release notes on the web page cover most of it, but here's a bit more detail on some of the more recent fixes: The Fedora bug turned out to be that none of the other shells were dynamically linked against libgcc_s.so.1, which the build was putting the TARGET version of in LD_LIBRARY_PATH before calling the shell during configure-target-libstdc++-v3 (to make libsupc++.a for uClibc++ to consume). But fedora's /bin/sh was. This didn't cause a problem when it was an OBVIOUS target binary and thus the dynamic linker wouldn't try to link in an arm binary x86_64 because the ELF constants were wrong, but an x86-64 binary did get linked in and then it died because uClibc wasn't installed on the host. The fix was to explicitly tell it SHELL=sh so it used busybox ash which is A) statically linked, B) never links against libgcc_s.so.1 even when it isn't. The arm target had several problems from 3.2 to 3.5, stemming from the fact that "arm versatile" apparently never worked on real versatile hardware, but only on qemu's emulation which apparently doesn't count. The upstream arm maintainers noticed some discrepances in the interrupt routing and fixed it to make the documentation instead of existing reality. So far nobody's found an actual arm versatile board to test this on; I patched the kernel code back to match what qemu expects. We can adjust qemu, but then new versions wouldn't be compatible with existing qemu releases. Me, I just want something that works, I don't care if it's a completely synthetic board that doesn't match any real hardware, the important thing is that the userspace code in the root filesystem works on real hardware, and that qemu can run that code. The ".config is a directory" problem with uClibc++ turns out to be that A) Garrett was using relative paths in a makefile, B) the root directory of some systems has a ".config" directory containing some random file that says "Trolltech". So when the makefile tries to include "../../.config" (which won't be there during the initial configure call, because we haven't created it yet), it does so using an include syntax that says "it's ok if we don't find this", but since it's not looking for an absolute path it falls back to a search path before giving up, and it starts with "/usr/include", and "/usr/include/../../.config" finds the one in the root directory, which is a directory, and it barfs. The fix: turn the relative path into an absolutepath when setting the variable, I used "shell readlink -f" on it. The sh4 bug is that the driver grew another port read that either works on real sh4 hardware or is a NOP (most likely a NOP), and qemu's emulation kills the emulator if a driver attempts to read from an unknown port. I just patched the kernel to read from a known port that will always return 0, the proper fix is to make qemu return 0 for this port instead of dying. Plus the usual stuff like rebasing the perl removal patches, I still haven't fixed Piotr's mdadm bug. It's not a kernel thing, it's a uClibc .configuration thing. I'll have to do another release for 3.6 anyway so it can go into that. Meanwhile, I'm playing with toybox and musl, and adapting Aboriginal to test both of those. Rob -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1345805133.0 From onefang at gmail.com Fri Aug 24 11:08:52 2012 From: onefang at gmail.com (David Seikel) Date: Fri, 24 Aug 2012 21:08:52 +1000 Subject: [Aboriginal] 1.2.0 is out! In-Reply-To: <50375B4D.6080501@landley.net> References: <50375B4D.6080501@landley.net> Message-ID: <20120824210852.4618cb1e.onefang@gmail.com> Congratulations, looked like a lot of little battles won. Though me actually trying this latest version will depend on me updating my qemu, which will depend on ... well, I got lots of my own little battles to get through. lol -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From rob at landley.net Fri Aug 24 12:09:45 2012 From: rob at landley.net (Rob Landley) Date: Fri, 24 Aug 2012 07:09:45 -0500 Subject: [Aboriginal] 1.2.0 is out! In-Reply-To: <20120824210852.4618cb1e.onefang@gmail.com> References: <50375B4D.6080501@landley.net> <20120824210852.4618cb1e.onefang@gmail.com> Message-ID: <50376F09.9000400@landley.net> On 08/24/2012 06:08 AM, David Seikel wrote: > Congratulations, looked like a lot of little battles won. Thanks. Sorry it took so long. > Though me actually trying this latest version will depend on me updating > my qemu, which will depend on ... well, I got lots of my own little > battles to get through. lol It should work with current qemu back to around 1.0. Most targets, anyway. 0.12.x and such, not so much. (Because qemu made incompatible changes to some of the board emulations where we can support before or after but not both.) Rob -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1345810185.0 From onefang at gmail.com Fri Aug 24 12:35:24 2012 From: onefang at gmail.com (David Seikel) Date: Fri, 24 Aug 2012 22:35:24 +1000 Subject: [Aboriginal] 1.2.0 is out! In-Reply-To: <50376F09.9000400@landley.net> References: <50375B4D.6080501@landley.net> <20120824210852.4618cb1e.onefang@gmail.com> <50376F09.9000400@landley.net> Message-ID: <20120824223524.52904390.onefang@gmail.com> On Fri, 24 Aug 2012 07:09:45 -0500 Rob Landley wrote: > On 08/24/2012 06:08 AM, David Seikel wrote: > > Congratulations, looked like a lot of little battles won. > > Thanks. Sorry it took so long. No worries, I know how that is. > > Though me actually trying this latest version will depend on me > > updating my qemu, which will depend on ... well, I got lots of my > > own little battles to get through. lol > > It should work with current qemu back to around 1.0. Most targets, > anyway. > > 0.12.x and such, not so much. (Because qemu made incompatible changes > to some of the board emulations where we can support before or after > but not both.) That's the problem, I'm currently stuck with the 0.12.3 qemu supplied with Ubuntu 10.04. Sometime later this year I'll be updating everything, including qemu. Got a few other things to get through before I can do that though. -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From rob at landley.net Fri Aug 24 13:55:07 2012 From: rob at landley.net (Rob Landley) Date: Fri, 24 Aug 2012 08:55:07 -0500 Subject: [Aboriginal] 1.2.0 is out! In-Reply-To: <20120824223524.52904390.onefang@gmail.com> References: <50375B4D.6080501@landley.net> <20120824210852.4618cb1e.onefang@gmail.com> <50376F09.9000400@landley.net> <20120824223524.52904390.onefang@gmail.com> Message-ID: <503787BB.2060705@landley.net> On 08/24/2012 07:35 AM, David Seikel wrote: > On Fri, 24 Aug 2012 07:09:45 -0500 Rob Landley wrote: > >> On 08/24/2012 06:08 AM, David Seikel wrote: >>> Congratulations, looked like a lot of little battles won. >> >> Thanks. Sorry it took so long. > > No worries, I know how that is. > >>> Though me actually trying this latest version will depend on me >>> updating my qemu, which will depend on ... well, I got lots of my >>> own little battles to get through. lol >> >> It should work with current qemu back to around 1.0. Most targets, >> anyway. >> >> 0.12.x and such, not so much. (Because qemu made incompatible changes >> to some of the board emulations where we can support before or after >> but not both.) > > That's the problem, I'm currently stuck with the 0.12.3 qemu supplied > with Ubuntu 10.04. Sometime later this year I'll be updating > everything, including qemu. Got a few other things to get through > before I can do that though. I just built qemu from source, which defaults to installing itself in /usr/local/bin, which is before /usr/bin in the search $PATH. You can even install the nightly snapshot du jour: git clone git://qemu.org/qemu cd qemu ./configure --disable-werror make -j 3 sudo make install And it should all "just work" without touching the existing qemu (which you can still access via /usr/bin/qemu). Not an option? Rob -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1345816507.0 From onefang at gmail.com Fri Aug 24 14:14:15 2012 From: onefang at gmail.com (David Seikel) Date: Sat, 25 Aug 2012 00:14:15 +1000 Subject: [Aboriginal] 1.2.0 is out! In-Reply-To: <503787BB.2060705@landley.net> References: <50375B4D.6080501@landley.net> <20120824210852.4618cb1e.onefang@gmail.com> <50376F09.9000400@landley.net> <20120824223524.52904390.onefang@gmail.com> <503787BB.2060705@landley.net> Message-ID: <20120825001415.338ab09d.onefang@gmail.com> On Fri, 24 Aug 2012 08:55:07 -0500 Rob Landley wrote: > On 08/24/2012 07:35 AM, David Seikel wrote: > > On Fri, 24 Aug 2012 07:09:45 -0500 Rob Landley > > wrote: > > > >> On 08/24/2012 06:08 AM, David Seikel wrote: > >>> Congratulations, looked like a lot of little battles won. > >> > >> Thanks. Sorry it took so long. > > > > No worries, I know how that is. > > > >>> Though me actually trying this latest version will depend on me > >>> updating my qemu, which will depend on ... well, I got lots of my > >>> own little battles to get through. lol > >> > >> It should work with current qemu back to around 1.0. Most targets, > >> anyway. > >> > >> 0.12.x and such, not so much. (Because qemu made incompatible > >> changes to some of the board emulations where we can support > >> before or after but not both.) > > > > That's the problem, I'm currently stuck with the 0.12.3 qemu > > supplied with Ubuntu 10.04. Sometime later this year I'll be > > updating everything, including qemu. Got a few other things to get > > through before I can do that though. > > I just built qemu from source, which defaults to installing itself in > /usr/local/bin, which is before /usr/bin in the search $PATH. > > You can even install the nightly snapshot du jour: > > git clone git://qemu.org/qemu > cd qemu > ./configure --disable-werror > make -j 3 > sudo make install > > And it should all "just work" without touching the existing qemu > (which you can still access via /usr/bin/qemu). > > Not an option? First I gotta get some sleep, then I gotta deliver the actual embedded project, that uses Aboriginal Linux, early next week. After those things are done, it can be time to experiment with new versions of qemu, Aboriginal Linux, and even the graphics library I use for it. A little late to be changing horses right now. The client wants another project after this one that uses this one as the base, so updating stuff after next weeks delivery is all good. On the other hand, I do intend to squeeze in an upgrade of my workstation OS to Ubuntu 12.04 soon after this project ends. So might be better to just do that first, and get an updated qemu for free. -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From rob at landley.net Sun Aug 26 19:23:22 2012 From: rob at landley.net (Rob Landley) Date: Sun, 26 Aug 2012 14:23:22 -0500 Subject: [Aboriginal] New aboriginal miss some headers? In-Reply-To: <5038EC19.30107@gmail.com> References: <5038EC19.30107@gmail.com> Message-ID: <503A77AA.8030901@landley.net> On 08/25/2012 10:15 AM, Piotr Karbowski wrote: > Yo, tried new aboriginal and while building busybox inside I got > > e2fsprogs/tune2fs.c:11:27: error: linux/ext2_fs.h: No such file or > directory As you noticed, that's a change to the linux kernel, new in 3.4. Specifically commit 4a165d25f63a989d0aabe9d8eed5b3a5d5da1862: ext2: No longer export ext2_fs.h to user space Since the on-disk format has been stable for quite some time, users should either use the headers provided by libext2fs or keep a private copy of this header. For the full discussion, see this thread: https://lkml.org/lkml/2012/3/21/516 Upgrading to current busybox fixes it. Rob -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1346009002.0 From brian.fix at gmail.com Fri Aug 31 02:08:03 2012 From: brian.fix at gmail.com (Brian Fix) Date: Thu, 30 Aug 2012 22:08:03 -0400 Subject: [Aboriginal] gfortran Message-ID: Hi Aboriginal Users, I found aboriginal looking into cross compiling python for my android. I've seen multiple threads and projects on how to do it, they usually involve a messy patch to apply to a specific version, and multiple stages of configuring and building with very long argument lists. Not the case with aboriginal, a simple: ./configure --without-threads --prefix=/home/python_blah make make install copied the results, and the dynamic libraries, onto my android device, ran elfpatch on the python to point the python to the new library locations, and i was done. Now I'm spoiled, I want numpy, scipy and matplotlib this easily! Unfortunately, numpy/scipy have dependencies which require a Fortran compiler, so here's my problem. I've downloaded the complete gcc tarball with gfortran and all other languages from their website and tried the same simple approach: tar xvzf gcc-4.6.0.tar.gz cd gcc-4.6.0/ ./contrib/download_prerequisits mkdir gcc_build cd gcc_build ../gcc-a.b.c/configure make but I get a build error in libgomp: checking whether make sets $(MAKE)... yes checking for armv6l-unknown-linux-gnueabi-gcc... /home/gcc_build/./gcc/xgcc -B/home/gcc_build/./gcc/ -B/usr/local/armv6l-unknown-linux-gnueabi/bin/ -B/usr/local/armv6l-unknown-linux-gnueabi/lib/ -isystem /usr/local/armv6l-unknown-linux-gnueabi/include -isystem /usr/local/armv6l-unknown-linux-gnueabi/sys-include checking for C compiler default output file name... configure: error: in `/home/gcc_build/armv6l-unknown-linux-gnueabi/libgomp': configure: error: C compiler cannot create executables See `config.log' for more details. make[2]: *** [configure-stage1-target-libgomp] Error 77 make[2]: Leaving directory `/home/gcc_build' make[1]: *** [stage1-bubble] Error 2 make[1]: Leaving directory `/home/gcc_build' make: *** [all] Error 2 (armv6l:1) /home/gcc_build # My question is multipart: should I be using gcc-4.6.0 or some other version more agreeable with aboriginal? Is there a better way to do this? I notices a series of gcc patches in the aboriginal-1.2.0.tar.gz, should I download 4.2.1 source apply these patches and compile? Are there any other easier ways of getting a Fortran on aboriginal linux? Thanks in advance for your time. Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob at landley.net Fri Aug 31 03:06:34 2012 From: rob at landley.net (Rob Landley) Date: Thu, 30 Aug 2012 22:06:34 -0500 Subject: [Aboriginal] gfortran In-Reply-To: References: Message-ID: <50402A3A.5080409@landley.net> On 08/30/2012 09:08 PM, Brian Fix wrote: > Hi Aboriginal Users, > I found aboriginal looking into cross compiling python for my android. > I've seen multiple threads and projects on how to do it, they usually > involve a messy patch to apply to a specific version, and multiple > stages of configuring and building with very long argument lists. > > Not the case with aboriginal, a simple: > > ./configure --without-threads --prefix=/home/python_blah > make > make install > > copied the results, and the dynamic libraries, onto my android device, > ran elfpatch on the python to point the python to the new library > locations, and i was done. > > Now I'm spoiled, I want numpy, scipy and matplotlib this easily! Yay! Glad you liked it. > Unfortunately, numpy/scipy have dependencies which require a Fortran > compiler, so here's my problem. > > I've downloaded the complete gcc tarball with gfortran and all other > languages from their website and tried the same simple approach: > > tar xvzf gcc-4.6.0.tar.gz > cd gcc-4.6.0/ > ./contrib/download_prerequisits > mkdir gcc_build > cd gcc_build > ../gcc-a.b.c/configure > make I have most of a Linux From Scratch 6.8 build at http://landley.net/aboriginal/control-images Basically if aboriginal linux's is fed a third hard drive image its init script will mount it on /mnt and then check for a /mnt/init and run that instead of a shell prompt. (Note the three second delay if you want to press a key to get a control prompt, then poke around in /mnt on the live system.) The native-build.sh wrapper plugs in a control image and then calls dev-environment.sh. I note that at one point I built the toolchains (it queries the host to see what the tuple is, ala "gcc -dumpmachine") but didn't install them because it would undo the distcc acceleration and the build was slow enough already. But that's probably a reasonable start? (Yeah, I should upgrade to the current Linux From Scratch. It's on my todo list. In the meantime, there's examples of natively building a gazillion packages, tweaked and tuned to install into aboriginal and gradually replace most of it...) > I notices a series of gcc patches in the aboriginal-1.2.0.tar.gz, should > I download 4.2.1 source apply these patches and compile? It's an option, but newer toolchains shouldn't need most of those patches. They're for uClibc and cross compiling and such. If _all_ you want is a fortran compiler, you shouldn't even need to replace the host toolchain. Just configure for fortran only. Dunno what kind of libc that might want, not familiar with it myself... Rob -- GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code. Either it's "mere aggregation", or a license violation. Pick one. 1346382394.0