[Toybox] Google Fucsia and Toybox

[Rob Landley]

tl;dr I followed the instructions and got magenta running under qemu,
and might poke at trying to build toybox for it.

On 08/17/2016 03:37 PM, Roberto A. Foglietta wrote:
> 
> 2016-08-17 18:11 GMT+01:00 Rob Landley <rob at landley.net
> <mailto:rob at landley.net>>:
> 
>     On 08/17/2016 01:47 AM, Roberto A. Foglietta wrote:
>     > Hi Rob,
>     >
>     > when you will have setup your fuchsia toolchain, could you help me in
>     > resolving the crt0.o linking issue in cross compiling.
> 
>     I'll see what I can do but I'm juggling a dozen balls right now. :)
> 
> 
> Ok, problem solved - errata corrige in attachment (only the broken part).
> :-)

Ok, Let's give it a try...

> ### FUCHSIA HOW TO RUN ###
> 
> ## Compiling and providing the buildtools
> 
> sudo apt-get install git libc-dev-bin gcc-multilib

and curl

> mkdir -p fuchsia
> pushd fuchsia
> git clone https://fuchsia.googlesource.com/buildtools
> git clone https://fuchsia.googlesource.com/magenta
> 
> INCL_DIR=$PWD/magenta/system/ulib/system/include

Export? Grep -r doesn't find it in any of the files downloaded
so far and I didn't spot it used in the later instructions...

(Do these instructions live somewhere online, or are these your
personal build notes?)

> pushd buildtools/
> ./update.sh

That's where I hit curl not found. (Installed it to get past that.)

Ninja _and_ cmake? Ok then...

> popd
> 
> ## Compiling and providing the toolchain for ARM target
> ## Available architectures: arm i386 aarch64 x86_64
> 
> sudo apt-get install texinfo libglib2.0-dev autoconf libtool libsdl-dev build-essential bison flex

Two of these packages serve no purpose.

Libtool exists to make non-ELF architectures behave like ELF, and
since Linux switched to ELF in 1996 libtool has had nothing to do
except screw up builds with unnecessary layers of indirection.

Seriously, you _don't_ want libtool in a modern system. Ever, if
you can help it. Any package that refuses to build on Linux without
libtool is a broken package. (MacOSX uses the mach-o binary format
instead of ELF, and Windows uses pe/coff, those may need libtool.
But on Linux there's nothing for it to do. The fact it often fails
to do nothing successfully, and breaks builds, is a trademark of
the Free Software Foundation.)

Info pages are also deeply obsolete. The FSF had somebody converting
them all to docbook masters in ~2003 (back when ESR was writing
doclifter). I dunno if they ever followed through, but that's how long
ago they admitted the FSF's bespoke not-invented here documentation
format based on the Gopher protocol had already failed.

> TARGET_ARCH="aarch64"

I thought you were doing x86-64? Ok, let's do that instead...

Are you exporting these?

> declare -i N_CPUS=$(egrep -c "^processor" /proc/cpuinfo)

A) Immutable? Does this get inherited by child processes?

B) We've had an "nproc" command for years, and it's in toybox.

C) You can "taskset 1 commandline..." to run in single processor mode
if something is using nproc.

> git clone https://fuchsia.googlesource.com/third_party/qemu

Any idea what's different between this and upstream qemu?

Hmmm, looks like you've merged the dtc and pixman subtrees into the
main repo, and/or disabled the dependency...

What would be really _really_ nice is if you could feed a compiled
device tree (.dtb file) to qemu and have it instantiate a board
emulation based on that description. There were people poking at this
at an ELC BOF in 2010 but I dunno what happened with it. (Grant Likely
would know.)

> pushd qemu
> git checkout fuchsia

You're aware you picked a project name nobody can spell on the first
attempt, right? I can only guess this is some strange response to the
Apple/IBM/Motorola "project pink" a couple decades back...

> ./configure --target-list=$TARGET_ARCH-softmmu --prefix=$PWD/qemu-runtime

You're installing it in the qemu source tree?

> make -j$N_CPUS install

Or just -j$(nproc)

> export PATH=$PWD/qemu-runtime/bin:$PATH
> popd
>
> SYSROOT=$PWD/buildtools/sysroot
> git clone https://fuchsia.googlesource.com/third_party/gcc_none_toolchains

As long as you're still using gcc rather than llvm, has anybody
introduced you guys to https://github.com/richfelker/musl-cross-make
yet?

Musl is BSD licensed, and ChromeOS is using musl-libc now, I believe,
so it's made it through the Google Lawyer Brigade somewhere. In
addition to being the default libc of Alpine Linux and such, it's been
ported to windows http://midipix.org/ and used on bare metal
http://ellcc.org/blog/?p=3247 so I strongly suspect it could be made
to work in fuschia without too much violence...

> ln -sf gcc_none_toolchains toolchain

Couldn't we have just checked it out there in the previous line?
> pushd toolchain
> ./doit -a $TARGET_ARCH -f -j$N_CPUS

Let's see, commented out the silly makeinfo check...

And hey, binutils broke because they STILL haven't fixed their
makeinfo test in autoconf. But I've had a (trivial) patch for
that for almost ten years now:
https://github.com/landley/aboriginal/blob/master/sources/patches/binutils-screwinfo.patch

Which I appended to gcc_none_toolchains/patches/binutils.patch.txt
and binutils built fine. (No, I won't assign a copyright to the FSF
for a ONE BYTE fix. And no, their stuff never seems to get fixed
unless someone outside the FSF does it for them and then shoves
the patch down their throat for a couple years. I am SO looking
forward to everything switching to llvm...)

And gdb died the same way. Ok, _that_ one I hit with a brick (patch
also attached, not the polite "fall back on failure" but "always make
zero length info files". Grrr. Why even have a ./configure if you
fail when what you expect isn't there? What's the POINT of a
configure stage in that case?).

For some reason I needed to rm .extracted-stamp in order for it to
re-extract the gdb directory? (This toolchain build infrastructure
seems a little on the brittle side.)

> popd
> TCBIN_PATH=$(ls -1d $PWD/toolchain/$TARGET_ARCH-*/bin)
> export PATH=$PATH:$TCBIN_PATH

Maybe just:

PATH="$(readlink -f $TARGET_ARCH-*)"/bin:"$PATH"

> ## Building Magenta for pc-x86-64 target
> ## Available build targets: 
> ##	magenta-pc-x86-64 magenta-qemu-arm32 magenta-qemu-arm64 pc-x86-64-test pc-x86-test qemu-virt-a15-test qemu-virt-a53-test rpi3-test
> 
> pushd magenta
> make -j$N_CPUS magenta-qemu-arm64

Huh, this is already using bits of musl.

You should definitely be using musl-cross-make for your toolchain
then. It would probably make your life a lot easier. :)

> ## Install qemu and make Magenta run with it
> # sudo -s
> # apt-get install qemu qemu-utils qemu-kvm qemu-system-x86 libvirt-bin bridge-utils
> # usermod -a -G libvirtd,kvm $(whoami)
> # rmmod kvm_intel kvm
> # modprobe -a kvm_intel kvm
> # exit

Little confused about the commented out stuff here?
We compiled a qemu from scratch. Why did we do that
if the host's version works?

> pushd build-magenta-qemu-arm64
> declare -i REAL_MEM_KB QEMU_MEM_MB N_CPUS_QEMU
> N_CPUS_QEMU=$[(N_CPUS+1)/2]
> REAL_MEM_KB=$(grep -e ^MemTotal: /proc/meminfo | tr -cd [0-9]) 
> QEMU_MEM_MB=$[($REAL_MEM_KB+1024)/2048]
> test $QEMU_MEM_MB -gt 512 && QEMU_MEM_MB=512
> qemu-system-$TARGET_ARCH -m $QEMU_MEM_MB -nographic -machine virt -cpu cortex-a53 -kernel magenta.elf -append ''

the kernel is actually in build-

Is there something wrong with just saying -m 1024?
(How much memory does this thing need, anyway?)

what does the -append '' do?

> # CPU type: arm926ej-s 
> 
>> ls /boot/bin
>> core-tests
>> thread-depth-test
>> dlog

Yay, it ran!

Is dlog like dmesg, or is it running another test? (Seems like the
latter.)

> # to terminate qemu
> CTRL-A X 

Haven't implemented shutdown yet, eh?

> popd
>
> ## Manual preparation of the current toolchain
> 
> export CC=aarch64-elf-gcc
> export CXX=aarch64-elf-g++
> export LD=aarch64-elf-ld.gold
> export AR=aarch64-elf-ar
> export AS=aarch64-elf-as
> export NM=aarch64-elf-nm
> export STRIP=aarch64-elf-strip
> export RANLIB=aarch64-elf-ranlib
> export DLLTOOL=aarch64-elf-dlltool
> export OBJDUMP=aarch64-elf-objdump
> export RESCOMP=aarch64-elf-windres
> export WINDRES=aarch64-elf-windres

Bulids use $CROSS_COMPILE for a reason...

> CFLAGS="-Wall -Wextra -ffunction-sections -fdata-sections -fPIC -mcpu=cortex-a53 -std=c11"
> CFLAGS="$CFLAGS -include config-global.h -include config-user.h"
> CFLAGS="$CFLAGS $(for i in $(find ../global ../system ../third_party -name include); do echo -n "-I $i "; done)"
> # export CFLAGS="$CFLAGS -I $SYSROOT/$TARGET_ARCH-fuchsia/include"
> export CFLAGS

*blink* *blink*

A while back I wrote my own version of the old uClibc cc wrapper:

  https://github.com/landley/aboriginal/blob/master/sources/toys/ccwrap.c

I tend to hit recalcitrant toolchains with that.

> export HOSTING_CRT0=./ulib/crt1.o
> 
> LIB_PATH_1=$(dirname $TCBIN_PATH)/lib/gcc/$TARGET_ARCH-elf/5.3.0
> # LIB_PATH_2=$SYSROOT/$TARGET_ARCH-fuchsia/lib
> # LIB_PATH_3=$PWD/ulib
> # export LIBRARY_PATH=$LIB_PATH_1:$LIB_PATH_2:$LIB_PATH_3
> # export LDFLAGS="-s -L $LIB_PATH_1 -L $LIB_PATH_2 -L $LIB_PATH_3"
> 
> LDFLAGS="-s -nostdlib -Lkernel -Lsystem -Lthird_party -z max-page-size=4096 --gc-sections"
> LDFLAGS="$LDFLAGS -z combreloc -z relro -z now -z text --hash-style=gnu --eh-frame-hdr"
> LDFLAGS="$LDFLAGS --build-id -pie -dynamic-linker ld.so.1 $HOSTING_CRT0"
> export LDFLAGS
>
> EXTRA_LIBS="$(find ./ulib/ -name \*.so.abi) $LIB_PATH_1/libgcc.a"

This seems somewhat elaborate.
 
> ## Compile and run your own code 
> 
> rm -f chilo.o chilo armstrong.o armstrong extra.bootfs
> $CC $CFLAGS -c armstrong.c -o armstrong.o
> $CC $CFLAGS -c chilo.c -o chilo.o -fPIC -Wno-unused-parameter
> $LD $LDFLAGS armstrong.o $EXTRA_LIBS -o armstrong
> $LD $LDFLAGS chilo.o $EXTRA_LIBS -o chilo

I missed a curve. Find . -name chilo.c hasn't got one.

> ## Remake the extra.bootfs initrd image
> 
> file chilo armstrong
> echo "bin/armstrong=./armstrong" > extra.manifest
> echo "autorun=./armstrong" >> extra.manifest
> echo "bin/chilo=./chilo" >> extra.manifest
> ../../buildtools/mkbootfs -o extra.bootfs extra.manifest 
> sync

In the magenta dir, docs/getting_started.md mentions this tool
and gives a little bit of instructions for it...

> ## Run a new qemu instance of Magenta with the new  initrd image
> 
> qemu-system-$TARGET_ARCH -m $QEMU_MEM_MB -nographic -machine virt -cpu cortex-a53 -kernel magenta.elf -initrd extra.bootfs -append ''

Still dunno what the -append '' does.

> Ciao,
> -Roberto

Rob
-------------- next part --------------
diff --git a/doit b/doit
index 0ed1f8b..0a4e717 100755
--- a/doit
+++ b/doit
@@ -140,10 +140,10 @@ if [ -z "$OUTDIR" ]; then
 fi
 ARCHIVES=$OUTDIR/archives
 
-if [ -z $(which makeinfo) ]; then
-    echo makeinfo not found. On debian/ubuntu this is provided by the texinfo package.
-    exit 1
-fi
+#if [ -z $(which makeinfo) ]; then
+#    echo makeinfo not found. On debian/ubuntu this is provided by the texinfo package.
+#    exit 1
+#fi
 
 export CC="cc"
 export CXX="c++"
diff --git a/patches/binutils-patch.txt b/patches/binutils-patch.txt
index 648ef13..5b77cbb 100644
--- a/patches/binutils-patch.txt
+++ b/patches/binutils-patch.txt
@@ -93,3 +93,21 @@ index 96c68de..bf25391 100644
  }
  
  // Print the SECTIONS clause to F for debugging.
+The binutils build notices that makeinfo is missing, but fails anyway, breaking
+the build.  Make it stop.
+
+The "info" file format is obsolete (similar to "gopher"), was never used
+by anyone but the FSF, and failed to even replace man pages (which are
+now available in HTML).
+
+--- binutils-2.18/missing	2005-07-13 20:24:56.000000000 -0500
++++ binutils-2.18/missing	2008-08-11 02:05:47.000000000 -0500
+@@ -299,7 +299,7 @@
+     fi
+     # If the file does not exist, the user really needs makeinfo;
+     # let's fail without touching anything.
+-    test -f $file || exit 1
++    test -f $file || exit 0
+     touch $file
+     ;;
+ 
diff --git a/patches/gdb-patch.txt b/patches/gdb-patch.txt
index bde0654..23270ef 100644
--- a/patches/gdb-patch.txt
+++ b/patches/gdb-patch.txt
@@ -13,3 +13,24 @@ diff -ru gdb-7.6/gdb/configure.tgt gdb-7.6.patched/gdb/configure.tgt
  x86_64-*-freebsd* | x86_64-*-kfreebsd*-gnu)
  	# Target: FreeBSD/amd64
  	gdb_target_obs="amd64-tdep.o amd64fbsd-tdep.o i386-tdep.o \
+--- gdb-7.10.1/missing	2015-02-19 05:58:08.000000000 -0600
++++ gdb.bak/missing	2016-08-19 15:46:23.001640243 -0500
+@@ -82,6 +82,18 @@
+ 
+ # If it succeeded, we are done.
+ test $st -eq 0 && exit 0
++if [ "$1" = "makeinfo" ]
++then
++  while true
++  do
++    [ -z "$1" ] && exit 0
++    [ "$1" = -o ] && break
++    shift
++  done
++  shift
++  touch "$1"
++  exit 0
++fi
+ 
+ # Also exit now if we it failed (or wasn't found), and '--version' was
+ # passed; such an option is passed most likely to detect whether the