[Toybox] Did I mention the release?

[enh]

On Mon, Mar 2, 2015 at 11:17 AM, Rob Landley <rob at landley.net> wrote:
> I uploaded toybox 0.5.2, with sed, printf, shred, base64, getenforce,
> setenforce, chcon, mix, and nsenter added to defconfig.

(so that this doesn't seem too extreme a thread-hijack, i should
probably say something about the release here, but i feel like toybox
is still in an early enough phase that i'm trying sync AOSP with
upstream roughly once a week whether or not there's a new version
number.)

> I see from
> https://android.googlesource.com/platform/external/toybox/+/master/Android.mk
> that android's using of "pending" commands has grown to:
>
>     toys/pending/dd.c \
>     toys/pending/expr.c \
>     toys/pending/getenforce.c \
>     toys/pending/hwclock.c \
>     toys/pending/more.c \
>     toys/pending/ps.c \
>     toys/pending/netstat.c \
>     toys/pending/route.c \
>     toys/pending/setenforce.c \
>     toys/pending/tar.c \
>     toys/pending/top.c \
>     toys/pending/tr.c \
>     toys/pending/traceroute.c \

this is _slightly_ misleading. you really want to look at the second
list in that file. the first list tells you what's built into our
toybox binary. the second list tells you what gets a symlink in
/system/bin. so it's stuff from pending that's on the second list we
_really_ need to worry about because those are the things people are
going to notice.

that said... anything on the first list but not the second is probably
particularly interesting to you because -- for one reason or another
-- they don't seem quite ready yet.

> So these are my current cleanup priorities. I'm currently poking at ps
> cleanup. (I also promised someone else to work on mdev this month.)

as a particularly observant follower of my AOSP gerrit activity might
discern, from the fact i committed a small improvement to the
_tool_box ps last week, i'm not expecting to switch to the toybox ps
in the near future.

there are two slightly conflicting -- because available time is small
and finite -- goals to choose from here, and it's perhaps worth
stating them explicitly. goal 1 is "move the maximum number of toolbox
tools over to toybox toys" and goal 2 is "have those toys that have
replaced tools be highly polished". for me, goal 1 is the primary
goal. a lot of the toolbox tools were really bad and almost any change
would have been an improvement; others (touch, say) had one thing they
did better that was easily ported to toybox, some had very unusual
behaviors (ps, say), and relatively few (dd being the sole remaining
example in toolbox) were of pretty high quality (in dd's case because
it's the NetBSD dd).

so for me i'd like to get the remaining SELinux tools turned into toys
and merged (last week's patches cover two of the three missing
commands, and then there are just the missing -Zs in existing
commands, such as ls), i'd like to fix the one glaring bug in dd (the
human-readable output bug i sent a patch for) and switch us over so
we've got as much time as possible to find any remaining bugs, and i
need to find the time to work out whether the toybox mount/unmount can
replace the toolbox ones or whether there's anything subtle but
important in the toolbox ones.

here's what's still in toolbox right now...

various things that toybox has, but not [necessarily] in a form we can
switch over to today:

toolbox/df.c
  missing -h; column alignment broken. (but no need to support old
Android output format.)

toolbox/upstream-netbsd/usr.bin/du/du.c
  our default block size was 512 (but people will probably be pleased
by that change); -h output slightly different. [patch sent upstream]

toolbox/ls.c
  missing -Z (-h would be nice, but toolbox doesn’t have that either)

toolbox/ps.c
toolbox/top.c
  output, options, and behavior all differ enough that any switchover
is going to be interesting. punt for now, worry about these later.

toolbox/renice.c
  ours has nonstandard -r, -t (equivalent to -n?), -g (“get”); do we need these?

toolbox/mount.c
toolbox/route.c
toolbox/umount.c
  work out whether toybox is superset.

three SELinux commands:

toolbox/load_policy.c (patch sent upstream)
toolbox/restorecon.c
toolbox/runcon.c (patch sent upstream)

the NetBSD dd implementation (patch sent upstream for the one obvious
bug i found in manual testing of the toybox dd):

toolbox/upstream-netbsd/bin/dd/args.c
toolbox/upstream-netbsd/bin/dd/conv.c
toolbox/upstream-netbsd/bin/dd/dd.c
toolbox/upstream-netbsd/bin/dd/dd_hostops.c
toolbox/upstream-netbsd/bin/dd/misc.c
toolbox/upstream-netbsd/bin/dd/position.c
toolbox/upstream-netbsd/lib/libc/gen/getbsize.c
toolbox/upstream-netbsd/lib/libc/gen/humanize_number.c
toolbox/upstream-netbsd/lib/libc/stdlib/strsuftoll.c
toolbox/upstream-netbsd/lib/libc/string/swab.c
toolbox/upstream-netbsd/lib/libutil/raise_default_signal.c

toolbox/uptime.c
  Android's output format is completely different (all about time
spent idle/sleeping, not concurrent users or processes); not clear
that merging makes sense.

various common-enough Linux tools that toybox doesn't have (should
look into turning these into toys instead, but that can wait):

toolbox/iftop.c
toolbox/ionice.c
toolbox/lsof.c

plus Android-specific things that i plan on leaving in a stub toolbox
for the time being (other than the fact i'd like to use the toybox
infrastructure in some of these, i'm not sure either side gains much
by having Android-specific stuff in toybox, but that's something we
can worry about when we've sorted out the things above):

toolbox/getevent.c
toolbox/getprop.c
toolbox/ioctl.c
toolbox/log.c
toolbox/nandread.c
toolbox/newfs_msdos.c
toolbox/prlimit.c
toolbox/r.c
toolbox/schedtop.c
toolbox/sendevent.c
toolbox/setprop.c
toolbox/smd.c
toolbox/start.c
toolbox/stop.c
toolbox/watchprops.c

i also plan on sticking with BSD grep for now (we can worry about the
quality of the BSD regex implementation in bionic and fleshing out the
missing grep features in toybox when everything else is settled):

toolbox/upstream-netbsd/usr.bin/grep/fastgrep.c
toolbox/upstream-netbsd/usr.bin/grep/file.c
toolbox/upstream-netbsd/usr.bin/grep/grep.c
toolbox/upstream-netbsd/usr.bin/grep/queue.c
toolbox/upstream-netbsd/usr.bin/grep/util.c

so -- though you didn't ask -- dd/df/du seem like priorities to me
because they seem close to being done, and the SELinux stuff because
they're just ports. there's a perverse incentive at play here too that
for me adding a new command (even if only to pending) will always be
the most urgent thing because it's always going to be easier for me to
cherrypick a fix than it is for me to switch to something "new". the
other incentive is that making a toy the default implementation means
we'll get to find what, if anything, actually causes trouble for
developers and can prioritize based on feedback.

 --elliott

 1426470470.0