[qcc] TODO?

[Rob Landley]

On 10/06/2015 02:35 PM, Sean Lynch wrote:
> Hi, everyone. I thought I'd unsubscribed from this list since there are
> no messages since May, but when I tried to resubscribe it told me I was
> already subscribed, and sure enough the archives show no messages since May.

Yeah, the project isn't really active at present.

> I started trying to use the "mob" branch of tinycc in a project,

There isn't a mob branch. There's a repostitory, but I haven't checked
stuff into it in years, for reasons explained on

http://landley.net/code/tinycc

I note that Bruce Ewing was doing some stuff with this, but he seems to
have wandered off again:

http://lists.landley.net/pipermail/qcc-landley.net/2015-May/date.html

> but so
> far I am not having very good success. In particular, it looks like
> there is not a way to use shared objects with TCC_OUTPUT_MEMORY without
> manually adding the symbols, which defeats a large part of my purpose,
> which was to have a runtime-compiled language that can directly use
> shared libraries by embedding C.

I'm spread a bit thin these days, currently most of my time goes to toybox.

Backstory: I've been working on what's now called Aboriginal Linux since
1999:

http://landley.net/aboriginal/about.html
http://landley.net/aboriginal/history.html
https://speakerdeck.com/landley/developing-for-non-x86-targets-using-qemu

It's a project to create the smallest linux development system capable
of rebuilding itself under itself, and building
http://linuxfromscratch.org under the result.

That project is was why I wound up doing so much work on busybox I
inherited the project (the previous maintainer, Erik Andersen, handed it
over so he could spend more time on buildroot). I wanted to replace all
the gnu tools with busybox (and glibc with uClibc, although these days
it's musl-libc.org).

I left busybox development due to a confrontation with Bruce Fscking Perens:

https://lwn.net/Articles/202106/
https://lwn.net/Articles/202120/

A little backstory: I cared about cleaning up the license because I was
launching the first GPL license enforcement suits with the SFLC after
inheriting the busybox "hall of shame" from Erik. It wasn't an academic
exercise, it was "we're about to sue people, let's get our house in
order". Bruce's "I have no enforceable IP claims but I own this anyway"
stance hit me so hard because I'd spent a couple years as a consultant
helping IBM's lawyers shoot down SCO's claims, and Bruce was making the
exact same "my code was once next to this code and the taint cannot ever
be removed bwahahaha mine mine mine" argument SCO had. The
http://busybox.net/~landley/forensics.txt style analysis I'd done to
fight off SCO may have helped win in court, but it couldn't get Bruce or
SCO to shut up and go away.

I continued the lawsuits for a year, but suing a dozen "gpl violating"
companies didn't add a single line of code to the busybox repository.
Having proved this effort was useless from an engineering perspective I
tried to shut it down, but although Eben Moglen stopped doing it, one of
the ex-FSF guys on his staff broke off and started the "software freedom
conservancy" to continue the lawsuits with other busybox copyright
holders, and eventually brought the FSF in to make a giant mess:

  http://landley.net/notes-2008.html#13-12-2008

During that "Bradley+FSF 4ever" lawsuit, I was consulting with Cisco to
make their newest router (um, wrt510n?) properly supported by open
source. The original 2003 legal saber rattling that gave us openwrt also
caused Linksys (then purchased by Cisco) to wash its hands of Linux for
5 years. They'd stopped all in-house development and outsourced their
linux stuff to taiwan, and switched new routers to a different OS. They
were now taking this mess back in house, upgrading all the package
versions, redoing the build system from scratch, and trying to do open
source _right_ this time. Bradley's lawsuit caused them to once again
drop Linux like a hot potato for 5 more years. Luckily, that recovery
period is finally up:

  https://lwn.net/Articles/660229/

But who really cares at this point?

The whole thing left a bad taste in my mouth, lawsuits are a bit like
nuclear weapons. Ok as a deterrent, but if you ever find yourself using
them it will just destroy everything on both sides and make lawyers
rich, and once an area has turned into toxic legal waste it's really
hard to clean it up again.

Meanwhile, I started toybox because I had a lot of half-finished command
rewrites I still enjoyed working on, and I kept at it on a purely
hobbyist basis for a few years. But I let the project roll to a stop
after a while because BusyBox had a 10 year headstart (much of which was
my own work), a dozen active developers, and was run by my hand-picked
successor. Toybox was fun to work on, but no matter how much technically
better I made the code, nobody would ever _use_ it. There wasn't enough
room between "busybox" and "perfect" to make it worth replacing, it was
"good enough" that there was no room left for something else in the same
ecological niche.

  http://lists.busybox.net/pipermail/busybox/2010-March/071783.html
  http://landley.net/notes-2011.html#31-03-2011

Then in 2011 Tim Bird approached me about his pet project of trying to
heal the split between Android and Linux. He had an "android upstreaming
project" that chipped off pieces of the android kernel, cleaned them up,
and submitted them to Linux. In the other direction, he wanted to create
a "bentobox" project that expanded Android's command line into something
more posix-like:

  http://www.elinux.org/Busybox_replacement_project

This coversation reminded me that although Android can't use GPL code
(they threw GPLv2 out with the GPLv3 bathwater, and had an official "no
GPL in userspace" policy which is why busybox predates android but still
doesn't ship with it). But that as copyright owner of toybox, I could
relicense it:

  http://landley.net/notes-2011.html#13-11-2011

And since my position on GPLv3 was that it was such a bad idea it would
probably eventually kill the entire GPL ecosystem:

  http://landley.net/notes-2011.html#16-12-2011
  http://landley.net/talks/ohio-2013.txt

Having toybox under a non-gpl license was actually a good thing.

So I restarted toybox with the explicit goal of turning Android into a
development environment, which ticked off the FSF something fierce
because they're idiots:

  https://lwn.net/Articles/478308/

They thought toybox was something Tim Bird created to stop the busybox
lawsuits, rather than something I'd been working on for years that I'd
realized I could repurpose to solve a problem I'd been thinking about
for a while:

  http://landley.net/notes-2010.html#09-10-2010

So I dove into toybox development with the explicit goal of making
android a self-hosting build environment. Since aboriginal linux already
_was_ a minimal self-hosting build environment, I knew exactly what was
needed and could make a roadmap:

  http://landley.net/notes-2010.html#09-10-2010

In 2013 I gave a talk explaining what I was trying to do:

  http://youtu.be/SGmtP5Lg_t0
  http://landley.net/talks/celf-2013.txt

And that eventually got in front of Elliott Hughes, the Android bionic
and toolbox maintainer, who started merging toybox into android:

  https://lwn.net/Articles/629362/

And that's where I am today, working as fast as I can to get toybox to
1.0 and turn android into a self-hosting development environment. Which
means getting AOSP to build on top of the result, which is its own giant
todo item, as explained:

  http://landley.net/aboriginal/about.html#selfhost
  http://landley.net/aboriginal/control-images/

I have a vague "I should work on qcc after toybox 1.0 and AOSP builds
under and aboriginal chroot", but it's far enough in the future...

> In general, the lack of ownership of tinycc makes me nervous about
> hitching my wagon to it.

Yeah, I washed my hands of it, with references:

  http://landley.net/code/tinycc

What initially attracted me to tinycc was:

  http://bellard.org/tcc/tccboot.html

And the windows clowns following grishka's "mob branch" (we don't need a
maintainer, we can have drive-by checkins with nobody steering!) are
_father_ away from making that work on a vanilla kernel than what that
shipped _eleven_years_ago_.

> But a fast, in-memory C compiler definitely
> seems ideal for my purposes. One of the many half-finished, abandoned
> code generation backends out there would potentially get me a JIT but
> without easy access to C shared objects. In that case, I might as well
> write my language using RPython and resign myself to manually writing
> wrappers or at best autogenerating them with a bunch of manual tweaking.
> 
> QCC seems like exactly what I'm looking for. I'd like to contribute,
> though I will need to go through a process to get my employer (Google)
> to disclaim copyright to any of my contributions that relate to their
> business (just once for the whole project, hopefully). I could also
> contribute financially through Bountysource or something if someone
> wants to set it up and that would give people extra time/incentive to
> work on QCC.

I think qcc is an awesome idea, and would love to see it through. But as
I explained at far more length than I expect you actually read above,
it's a bit far down on my todo list.

LLVM (http://ellcc.org and similar) can make a usable toolchain to get
GPLv3 gcc out of the picture. In the long term having our only C
toolchain be written in C++ is _insane_, but http://pcc.ludd.ltu.se/
(despite initial promise ala https://lwn.net/Articles/255558/ ) is
moribund, and libfirm/cparser can't build much real-world stuff yet either.

> It would be helpful if someone (I assume that would be Bruce since Rob
> isn't listed as a contributor) could write a TODO file that's a little
> more amenable to consumption by someone who's not yet familiar with the
> codebase than the XXX and HIHI(?) comments and TODO section in qcc.c.
> Such a file would double as a status report for anyone (like me)
> following development.

I would love to contribute _to_ such a project, time permitting.
Although my first priority would be license clearance, ala:

  http://landley.net/notes-2012.html#14-06-2012

I went to all the trouble of getting the toybox "zero clause bsd"
license approved by spdx, so it would be nice to do qcc under that.

> Thanks for working on this! This is a very underserved space since gcc
> or LLVM work "well enough" for most people. Actually they work
> fantastically as long as it's not runtime code generation you're
> interested in.

As I mentioned in the 2013 "why is toybox" youtube video mentioned
above, a self-hosting development environment needs 4 things:

1) kernel
2) libc
3) command line
4) compiler

Toybox is #2, musl-libc is #2, linux is our only #1 at present, although
long-term linux has a problem (think
http://www.zdnet.com/article/graying-linux-developers-look-for-new-blood/ isn't
an issue? Read the "there is no problem" comments on
https://lwn.net/Articles/659221/ and note the closest thing to an
official response was https://lwn.net/Articles/659884/ ).

QCC was my solution to #4. I would love to see somebody do a
busybox-like compiler right. Public domain licensed, capable of building
the other 3 parts as well as itself, and implementing "make" and "nm"
and "objdump" as commands it knows how to do. (Turns out the kernel
build needs 'em to generate headers and such. Oh, and building busybox
or toybox requires dead code elimination to work. I already fixed
constant propogation as required by c99...)

Rob

 1444762899.0