[Toybox] Can not cross compile on Mac OS X (Mavericks)

[Rob Landley]

On 01/07/2016 05:42 AM, Raphael Cohn wrote:
> Originally raised as GitHub issue #17
> 
>   * |make menuconfig| works

Wow. That was the big blocker for me every time I tried to get it to
work on one of the BSDs:

  http://landley.net/notes-2014.html#18-06-2014

I have a todo item to write my own menuconfig implementation, which
would incidentally remove the last piece of GPL build infrastructure (we
don't ship the binaries, it just produces a .config text file we then
consume, but having it in the build at all uncomfortable). But that
comes after vi and top and less (using the not-curses infrastructure I
started developing in hexedit).

>   * |make| fails because a |sed| expression isn't understood by Mac OS X
>     (and hence BSD) |sed| It may be possible to have a different
>     expression. If you can give me a little insight into what the
>     expression is trying to do, I can have a go at sorting this out.

Creating a generated/sed that the build can use is a pending todo item
dating back to 2014:

  http://landley.net/notes-2014.html#28-11-2014

Toybox sed can build toybox, but building toybox sed requires
generated/* files, many of which are created with sed. It's a chicken
and egg problem. (Not unsolvable, but not trivial either. And as I said,
it hadn't become the bottleneck either.)

> I'm really keen to get a minimal toybox working on other POSIX
> platforms, because it provides an excellent baseline for a consistent,
> secure and sensible set of tools to enable cross compilation toolchains.

I was trying to get a BSD build environment set up for a long time. (I
tried freebsd, netbsd, and pcbsd, having asked everyone from Kirk
McKusick to Randy Harper for help along the way.)

The cyanogenmod guys have apparently been checking in macosx patches,
which I've pulled a tree of. They added an #ifdef APPLE to toys.h when
I've gone to a lot of effort to confine all that stuff to lib/portability.h.

This unfortunately implies that a lot of the LSB stuff should move to
lib/portability.h, which... is a definitional issue. So far this has
been producing a Linux command line, on the theory that the Intel Binary
Compatability Standard collapsed in favor of "just run Linux binaries",
the 86open standards effort also declared linux binaries the de-facto
standard and dissolved without trying to create a new one, sco created
lxrun which solaris ported and extended, and all that was back in the 90's.

These days AIX has a Linux emulation layer, FreeBSD and NetBSD have
linux emulation layers... MacOS X _not_ having one is a _choice_ they made.

That's why I let Linux Standard Base headers live in toys.h instead of
portability.h. Any command that needs to include linux/*.h or asm/*.h
has to do it in the command (so that command can be switched off in
places it can't build, and yes you need those to do things like ldconfig
and no block copying a structure from the header into your program is
not an improvement). But I'm assuming basic Linux compatibility (mostly
circa 2008 or so, since we depend on posix-2008) in at least an emulated
environment.

If you're going to argue "LSB 4.1 circa 2011 is too new, you need to
bump back to LSB 4.0 circa 2009 since that's about the posix-2008 level"
I'd consider that a fairly strong argument. But what the cyanogenmod
guys did is:

https://github.com/CyanogenMod/android_external_toybox/blob/49a8567012a/toys.h#L64

The first header that chops out is pty.h which was in LSB 3.0 back in
2005, see
http://refspecs.linuxfoundation.org/LSB_3.0.0/LSB-Core-generic/LSB-Core-generic/libc-ddefs.html
section 1.4.23. That's plenty of time to decide to be compatible with
it, or not. They decided not.

It's not just headers. On the #musl channel on freenode I had a long
discussion with the midipix maintainer about which /proc and /sys files
toybox uses. I.E. I dug through the code to get him a list so he knew
which files to emulate. It's not a static list, but I need to expand the
test suite (and fix it so all the existing tests pass) so people can
regression test this sort of thing and spot what's missing.

Some portions of the code have implicit Linux assumptions. For example,
are explicitly using Linux ioctl numbers, such as this bit of ifconfig.c:

https://github.com/landley/toybox/blob/master/toys/other/ifconfig.c#L468

Where we mask the known SIOCSI ioctl value range, because it doesn't
vary by target and changing it would break backwards compatibility with
existing Linux binaries (which is against linux-kernel development
policy) so we can rely on it not changing.

> Oh, and not using autocruft gets it an extra star...

I have... opinions on that:

http://lists.landley.net/pipermail/aboriginal-landley.net/2011-June/000859.html

http://lists.landley.net/pipermail/aboriginal-landley.net/2011-June/000860.html

> More details:-
> 
> maclaptop:toybox raph$ make
> scripts/make.sh
> Generate headers from toys/*/*.c...
> Make generated/config.h from .config.
> generated/flags.h sed: 2: "s/" *"//g;/^#/d;t clear ...": undefined label 'clear;:clear;s/"/"/p;t;s/\( [AB] \).*/\1 " "/p'
> sed: 1: "s/ A / /;t pair;h;s/\([ ...": undefined label 'pair;h;s/\([^ ]*\).*/\1 " "/;x;b single;:pair;h;n;:single;s/[^ ]* B
> //;H;g;s/\n/ /;p'
> clang: error: unable to execute command: Broken pipe: 13
> clang: error: clang frontend command failed due to signal (use -v to see invocation)

The fix is for me to add a build_sed target that creates a static
generated/sed the rest of the build can use. Lemme see what I can do...

Thanks,

Rob