[Toybox] awk (Re: ps down, top to go)

[Rob Landley]

On 05/30/2016 03:23 AM, Andy Chu wrote:
> This argument doesn't really matter to me though... I'm not trying to
> convince you to follow POSIX for your shell.

I plan to largely follow posix. My point is that if bash and posix have
disagreed on something 20 years, posix didn't matter enough to get the
bug fixed in bash, so I probably don't have to care much either.

My main objection to posix are that the committee seems to have gotten
paralyzed sometime last century. It still maintains its source in sccs.
It removed cpio despite Linux basing RPM and initramfs on the -h newc
variant (and yet the 2013 update still expects "file" to recognize cpio
files?). It standardizes "pax" instead of "tar".

I _want_ posix to be a better standard than it is. I wanted LSB to be a
better standard than it was, too. :(

>> I never look at FSF code. On general princples. But the behavior of the
>> standard Linux command line is what Linux developers (and the build
>> systems they write) expect.
> 
> I think build systems significantly more constrained problem, because
> that is the one place where people care if their shells scripts are
> portable.

Where _some_ people care about that. I've built an awful lot of packages
that... really don't.

> Probably 75% to 95% of major packages are portable to BSD (without
> installing bash on BSD).

I'm currently staring at a patch to make toybox build on freebsd. Alas,
I have questions, and the guy who sent me the link has never emailed me
and is only intermittently on the IRC channel... :(

> If a package uses autoconf, then it will
> very small subset of POSIX shell.

That is _so_ not an endorsement of autoconf.

> And even programs with hand-written
> configure scripts like QEMU obviously are not using every nook and
> cranny of bash in them.

QEMU goes to great lengths to build on Windows, which isn't even LP64.
(And yet they use libglib.)

>>>> Keep in mind, over the years people have written a dozen different
>>>> shells. It's really not that big a deal, I just want to do it _right_ so
> 
> The claim that there are dozens of shells didn't seem suspect to me at
> first...  There are tons of Unixes that I've never used, both now and
> in the past.
> 
> But now that I've done more research, I believe there have only been 4
> open source Bourne/POSIX-compatible shells EVER.

If you're restricting it to Bourne compatible, you're cutting out things
like Bill Joy's csh and plan 9's "rc"...

> And only 2 of them were started with unpaid labor.  They are:
> 
> - Almquist shell (dash, NetBSD ash, busybox default sh)
> - zsh (arguably not posix compatible, but it can be made posix
> compatible by setting lots of flags)
> - bash

The Bourne Again shell was not the original Bourne shell, so I'm already
confused by your selection criteria...

> - Korn shell and derivatives (pdksh, mksh)  Solaris uses a derivative
> of Korn shell.

Um, ksh was based on the unix 7 bourne shell? So this is where bourne
slots into your taxonomy...?

For IP reasons, Minix and Coherent had their own shells written from
scratch. In busybox lash and hush were fresh rewrites (and hush is
reasonably usable on nommu, ash doesn't build for that). I believe david
bell wrote sash from scratch. There are _several_ different craptacular
shells in uclinux (nwsh is 775 lines of C and msh "minimal shell" is
_53_ lines of C. Through use of horrible macros. But it has pipe and
redirect!)

Plus other oddballs like
https://en.wikipedia.org/wiki/Friendly_interactive_shell and
https://en.wikipedia.org/wiki/BeanShell and
https://en.wikipedia.org/wiki/Scsh and so on. A friend of mine's website
used to give you a shell prompt, implemented in javascript...

> bash was started with the paid labor of Brian Fox, since apparently
> Stallman was impatient with another volunteer effort.

I care that Linux used it, not what Stallman wanted. (I believe Linus
first started distancing himself from Stallman after the two met at an
O'reilley conference in 1996.)

> And Korn shell
> was open sourced in 2000, but its development was paid for by AT&T.

So was the original Thompson shell, and the Bourne shell that replaced
it in Unix version 7, and Plan 9's rc...

> So my claim is that there is NO other distinct open source code
> lineage which is even roughly POSIX compatible.

*shrug* So I'll make one?

> It takes 5,000-10,000 LOC to make a POSIX compatible shell,

Nah.

> and you won't find that amount
> of original code in any other implementation.  All shell variants are
> pushing around ancient code from very few lineages.
> 
> Does anyone have evidence to the contrary on this point?
> 
> Moreover, I don't think there are any other commercial code lineages
> besides the *original* Bourne shell, but I'm less confident about that
> claim.

The first commercial Unix clone shipped in 1980:

  https://en.wikipedia.org/wiki/Coherent_(operating_system)

Dennis Ritchie was sent by AT&T to audit it, and confirmed that there
was no AT&T code in there.


https://groups.google.com/d/msg/alt.folklore.computers/_ZaYeY46eb4/5B41Uym6d4QJ

Minix was started around 1983 when AT&T forbid the use of Unix version 7
source in operating system courses. (Lots of people stuck with the Lions
book describing Unix version 6, but Tanenbaum wrote his own clone from
scratch. Linux forked off of Minix in 1990, but as a reboot rather than
incorporating the old code.)

There were lots of other lineages too, many lost to history in AT&T's
great System V push, which also gutted a lot of the BSD use. (That's why
IBM's AOS (https://en.wikipedia.org/wiki/IBM_Academic_Operating_System)
got rebooted as AIX, and SunOS got rebooted as Solaris: because AT&T was
pushing people to use its System V intead of BSD, and using vague legal
threats to do it, which Berkeley's Computer Science Research Group
eventually successfully fought off in
https://www.bell-labs.com/usr/dmr/www/bsdi/bsdisuit.html but that took
until 1993 to wrap up, by which point Linux was 2 years old.)

This of course predated posix by many years, but it shows that there's a
lot of different taxonomies out there.

(Did I mention computer history is a hobby of mine?
http://landley.net/history/mirror)

> Andy

Rob