[Toybox] Fwd: Re: Is binutils neccessary?

Mon Dec 15 11:02:10 PST 2014

Stephen Turner wrote:
> On Dec 13, 2014 4:32 PM, "Rob Landley" <rob at landley.net> wrote:
> >
> > Mind if I cc: the toybox or aboriginal list on the reply to this?
> >
>
> Please feel free to do so any time you feel appropriate.

-------- Original Message --------
Subject: Re: [Toybox] Is binutils neccessary?
Date: Sat, 13 Dec 2014 15:33:29 -0600
From: Rob Landley <rob at landley.net>
To: stephen Turner <stephen.n.turner at gmail.com>

On 12/12/2014 01:08 PM, stephen Turner wrote:
> 
> 
> On Fri, Dec 12, 2014 at 1:15 PM, Rob Landley <rob at landley.net
> <mailto:rob at landley.net>> wrote:
> 
>     On 12/11/14 14:04, Isaac Dunham wrote:
>     > By the time we have a full-featured depmod, nm would be not much more
>     > work.
>     > A full POSIX ar (adequate for libraries) would also cover ranlib.
>     > strings is already done.
>     > size and strip are the main binutils tools in addition to those...
> 
>     It's possible that binutils (ELF-only) is indeed within scope of toybox.
>     Not for the 1.0 release, but adding it on later: ELF is a fairly
>     standardized archive format? Sort of?
> 
>     Modulo all the darn target dependencies, but then I haven't decided
>     strace isn't in scope for toybox either, and that's nothing BUT target
>     dependencies. (Again: post-1.0.)
> 
>     But it's more in scope for qcc, and either way ONE of the two projects
>     needs a "make" implementation which is more work than the linker. And I
>     _do_ know that a C compiler is definitely out of scope for toybox. (Yes,
>     http://pubs.opengroup.org/onlinepubs/9699919799/utilities/c99.html is a
>     posix command. I don't care, it's out of scope. Even tinycc hardwired to
>     output 32 bit x86 code only was 100k of source, and I may be good at
>     programming as haiku but I'm not better than Fabrice Bellard...) So qcc
>     has to be a separate project...
> 
> 
> I agree qcc only makes sense as a separate project. its a dev
> environment as opposed to an OS. PCC is bsd licensed and from what i
> understand c99 compliant. it certainly works better than TCC as far as
> sucessful compiles on existing GNU and musl code.  Also i believe a good
> portion of the PCC project is scripted to lex and yacc which could be
> included in either qcc or toybox as seen fit.

PCC is nice, but A) lex and yacc as prerequisites, B) continuing
dependence on binutils mean my goal of getting a self-bootstrapping
system down to 4 packages ain't happening unless I implement more than
the whole of binutils in toybox.

If I'm grabbing tinycc's linker anyway, I might as well grab the rest of
tinycc.

The next problem is target support: qemu has a full time staff of people
maintaining support for a couple dozen targets and target variants. You
want to generate code for armv4tl but not use the armv5 instructions?
They can do that, it's built into tcg, maintained, and regression tested
by a flood of existing users. They've even got ELF loaders and such that
work on that target already.

PCC? Maybe they support powerpc? Ish?

>     On the other hand, I might add binutils bits to toybox for a while, and
>     then when qcc starts up go "all this is moving to qcc" and yank it back
>     out of toybox with a pointer to the other project? I dunno. I wouldn't
>     want users to lose funtionality but spliting a large package into
>     smaller packages has precedent. And keep in mind qcc is designed as one
>     binary that acts like cc, ld, cpp, as, nm, objdump, and so on depending
>     on what symlink you call it through, just like toybox.
> 
> 
> Right, and im excited about the idea of qcc.

I have _no_ time to work on it until tcc goes the way of aboriginal
(feature complete, in maintenance mode).

Of course there's a bunch of new stuff I'd like to do with aboriginal
too. I need to finish the musl conversion, update bootstrap-lfs to 7.4
or 7.6 _with_ beyond linux from scratch, get bootstrap-debian
implemented (well, probably devian) and make another stab at
boostrap-funtoo, and it should generate an initramfs of
simple-root-filesystem always and have the native-compiler stuff be the
hda.img squashfs, with the init script symlinking and bind mounting
stuff into place in the initramfs so it works...

But toybox takes priority.

> I would say that including
> binutils initially into toybox might make sense for your aboriginal
> build and in general considering qcc will take time to polish off and
> you will want to make use of binutils if you have it completed well
> before qcc is ready.

Supporting PCC is one argument for binutils in toybox, yes. And
including lex and yacc. (lex is at least a posix command, I'm halfway
convinced I should include that anyway. It's not just in the roadmap
because it's not _required_ for any of the current use cases. LFS
bootstrap can build an external lex under aboriginal.)

> Perhaps in this case its best to keep the binutils
> code in the testing setion of toybox and not included in precompiled
> binaries? this way its available for use and testing while still using
> gcc to ensure no bugs are present while building out qcc.

If somebody, somewhere, can build a linux kernel with pcc (even a mostly
allnoconfig one with extensive patching applied, ala the old tccboot
iso), I'll take a second look at it.

Last I checked it was down to one part-time developer who only cared
about bsd and not linux, but I'm told it's revived a bit since then...

>     I also note that qcc's linker should work with pcc and llvm (and even
>     gcc, if that's still around by then). If you want to use just part of
>     the project, that's ok. So "lives in toybox" vs "lives in qcc" is less
>     of a bright line there...
> 
> 
> you mentioned that toybox would someday be able to build into multiple
> binaries or at least that was one idea,

It does it now. "make change" or "scripts/single.sh commandname".

Not all of the commands build standalone yet, and a few of the ones that
compile are broken. I have a plan to tackle that, which involves redoing
some of the generated directory creation:

http://landley.net/notes-2014.html#18-10-2014
http://landley.net/notes-2014.html#05-11-2014
http://landley.net/notes-2014.html#28-11-2014

> would it be a reasonable idea
> then to have the qcc/binutils/make included in toybox but at compile
> time built into a separate binary?

Not specially. It can "default n" or "default y", that's pretty much it
in terms of distinguishing a command. (The
posix/lsb/other/examples/pending directories are largely cosmetic. They
let developers know stuff but don't have much impact on the actual build.)

>     But again: it's not in the toybox 1.0 roadmap, and qcc doesn't start
>     before then unless somebody else does it for me. :) (Take my project
>     idea, please...)
> 
>     Rob
> 
> 
> 
> Rob, if i could code i would gladly start on the qcc project. I'm
> learning by reviewing code while trying to build out my musl project but
> i'm a far cry from a programmer that's for sure.

Everybody has to start somewhere. :)

> Perhaps once i get my musl environment working the way i want i could
> see if PCC could be adapted for this purpose, it feels like its farther
> along. I need to find out about the multiple licenses first though.

PCC has multiple licenses? (I haven't looked into it at all recently...)

> Also
> on the TCC list someone mentioned a C++ to C translator that may be of
> use with PCC to extend its capabilities. 

I don't care about the tcc list, the project died until my fork revived
interest in it (and then I stopped my fork and the other one rolled to a
stop, and I started up again and it started up again... THREE TIMES
before I gave up). Some people insisted that the CVS was "official" in a
way my repository wasn't, so lwn.net would cover their releases but not
mine, and eventually a windows developer who doesn't understand (or care
about) Linux took over as the new "maintainer" in that he did a "mob
branch" where he basically accepted all patches from everybody without
even reviewing them to see if they compiled.

Honestly, they were following me around while not understanding what I
was doing to a level I found outright annoying. After I got permission
from Fabrice to relicense his old code and started doing the forensic
analysis to clean out external contributions up to the point my one
forked off, this happened:

http://lists.nongnu.org/archive/html/tinycc-devel/2013-05/msg00015.html

Yes, that's the current tcc maintainer. No, he didn't understand that he
CAN'T just arbitrarily relicense code he's been collecting in a "mob
branch" for several years where he didn't even record who contributed
each patch. (Oh great, you've got an alias and a lavabit email. The NSA
shut that service down a while ago now. Very useful...)

And yes, he posted that message to the thread in May, despite me posting
this to the thread the previous month:

http://lists.nongnu.org/archive/html/tinycc-devel/2013-04/msg00064.html

Note that the thread topic change was in this message:

http://lists.nongnu.org/archive/html/tinycc-devel/2013-04/msg00052.html

That's Grischka replying to me, but removing my name from the snippet he
replied to...

Sorry, pet peeve. My plans for qcc are to take up from where my fork let
off. I have no interest in any code the tcc mailing list has added since
Fabrice left.

Rob

 1418670130.0