[Toybox] ps and top (and Android)

[enh]

On Sat, Apr 23, 2016 at 3:06 AM, Rob Landley <rob at landley.net> wrote:
> On 04/21/2016 08:59 PM, enh wrote:
>> On Thu, Apr 21, 2016 at 1:06 AM, Rob Landley <rob at landley.net> wrote:
>>> On 04/20/2016 08:47 PM, enh wrote:
>>>> re: https://github.com/landley/toybox/commit/529d5db29a32db565d2ee0aeac133e47ec045e12,
>>>> note that we're not switched over because neither ps nor top have
>>>> thread support. everything on Android is threaded except init, so
>>>> people would notice :-)
>>>
>>> I was tracking whether or not it was in pending, but you're right. My bad.
>>
>> well, it's your roadmap --- you get to choose what it means :-) i just
>> wanted to point out that we might have been interpreting it
>> differently.
>
> Indeed, but you maintaining a toybox roadmap provides me with
> information I wouldn't otherwise have. (I can scribble my plans down in
> a textfile, and they're subject to change anyway.)
>
> (I still occasionally check https://wiki.tizen.org/wiki/Toybox but other
> than typo fixes in January it hasn't been touched in two years. I
> haven't figured out how to follow cyanogenmod's toybox discussions yet...)
>
>>> Yesterday I found out I'm going to Japan on Sunday, so the release I was
>>> trying to prepare at the end of the month got bumped up. I have a 9 hour
>>> layover in San Francisco on Sunday, and I'm hoping to upload it then.
>>>
>>> I have like 6 bugs I need to fix between now and then (the find
>>> segfault, the new bzcat segfault, expr memleak, etc). Plus I'm in the
>>> process of cleaning up another half dozen commands out of pending, and I
>>> should get at least a couple of them closed out and promoted this release.
>>
>> barring anything huge enough to warrant a cherrypick, what you see in
>> N DP2 (http://developer.android.com/preview/index.html) should be what
>> ships in N as far as toybox goes.
>
> Going by tags that's commit 9fcaca8434ec for android-n-preview-2 (modulo
> the build system churn). Check that out in the android repo, make
> oldconfig, switch off selinux support, build, and:
>
> acpi base64 basename blkid blockdev bunzip2 bzcat cal cat chattr chgrp
> chmod chown chroot cksum clear cmp comm cp cpio cut date dd df dirname
> dmesg dos2unix du echo egrep env expand expr fallocate false fgrep
> find flock free freeramdisk fsfreeze grep groups head help hostname
> hwclock id ifconfig inotifyd insmod install ionice iorenice kill killall
> ln logname losetup ls lsattr lsmod lsof lsusb makedevs md5sum mkdir
> mkfifo mknod mkswap mktemp modinfo more mount mountpoint mv nbd-client
> nc netcat netstat nice nl nohup od partprobe paste patch pgrep pidof
> pivot_root pkill pmap printenv printf ps pwd pwdx readlink realpath
> renice rev rfkill rm rmdir rmmod route sed seq setsid sha1sum sleep
> sort split stat strings swapoff swapon sync sysctl tac tail tar taskset
> tee time timeout top touch tr traceroute traceroute6 true truncate
> tty ulimit umount uname uniq unix2dos uptime usleep vconfig vmstat
> wc which whoami xargs xxd yes
>
> (Woo, 149 commands!)
>
> Pending commands included in the build:
>
> $ make list_pending | tr ' ' "\n" | \
> egrep "^($(./toybox | tr '\n ' '||'))\$" | xargs
> dd expr lsof more netstat route tar tr traceroute traceroute6
>
> Non-pending commands _not_ included in the build:
>
> $ make list_working | tr ' ' "\n" | \
>   egrep -v "^($(./toybox | tr '\n ' '||'))\$" | xargs
>
> $ make list_working | tr ' ' "\n" | \
>   egrep -v "^($(./toybox | tr '\n ' '||'))\$" | xargs
> catv chvt count eject factor fstype fsync halt hello hexedit hostid
> iotop killall5 link login lspci mix mkpasswd nfsmount nproc nsenter
> oneit passwd poweroff readahead reboot reset shred skeleton
> skeleton_alias su switch_root test_many_options unlink unshare uudecode
> uuencode w who
>
> what have we skipped:
>
> You have your own init/login and toybox hasn't completed that circle yet
> anyway (halt/oneit/poweroff/reboot/reset/switch_root/killall5), you
> don't use standard uids (login/mkpasswd/passwd/su/w/who), no vga ttys
> (chvt), I should filter the examples directory out of "list_working"
> (hello/skeleton/skeleton_alias/test_many_options), nfsmount is just a
> todo item in mount (it needs a command that can prompt for password),
> and you have your own non-oss audio layer (mix).
>
> We can also say that count/factor aren't important, blkid covers fstype,
> sync covers fsync (albeit less efficiently, and I'm trying to remember
> if the "fsync /dev/sdb" to flush pending I/O to a specific block device
> is actually supported upstream or was an out of tree patch...), and you
> have the standard cat -v (catv).
>
> That leaves:
>
> eject hexedit hostid iotop link lspci nproc nsenter readahead shred
> unlink unshare uudecode uuencode
>
> The four that jump out at me are link, unlink, uuencode, and uudecode,
> which are in posix. I'm aware ln and rm cover the first two, and base64
> more or less covers the rest, and yet:
>
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/link.html
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/unlink.html
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/uuencode.html
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/uudecode.html
>
> eject: is that actually used for anything but CD/DVD drives these days?
> I suppose it's really a special case of the ioctl...
>
> hexedit: perhaps an acquired taste. (And I still need to work out how to
> make it insert and delete, and let you cursor over into the ASCII
> display part and type stuff in there...)
>
> hostid: maybe not relevant in context? Large Company's developer
> submitted it and it should REALLY be showing ipv6 addresses these days
> instead of ipv4. Its standardness is dubious...
>
> iotop: If the android kernel doesn't have CONFIG_IRQ_TIME_ACCOUNTING
> enabled, this won't do you a whole lot of good. :)
>
> lspci: most android devices don't have pci busses, but android gets used
> in the embedded space sometimes... *shrug* (I worked on one that did in
> 2012, but the product was doomedish and the company recently got acquired.)
>
> nproc: prints number of CPUs. Might be useful, and is in modern
> coreutils, but you can get the info out of /proc or /sys if you need to.
>
> nsenter/unshare: Someday, opening the containers can of worms would be a
> really nice thing for Android to do. But we've talked about how there's
> Much Design Work to do to make that happen. Still, if the kernel
> plumbing's enabled, maybe for O?
>
> readahead: was requested. Seems like a redhat-ism to me.
>
> shred: was also requested. Probably useless on log based flash
> filesystems, or even stuff like btrfs that can snapshot (let alone have
> multiple heads).
>
>> i'm on the opposite end of my
>> release cycle now, where now's a great time to make big changes in
>> AOSP master, for O next year. which is why my mind's back on ps and
>> top.
>>
>> dd too, though as i said last week i plan on waiting for that to leave
>> pending because (unlike expr) it's heavily used.
>
> Gotit: ps, top, and dd are android todo priorities. I also have mtools,
> genvfatfs, mke2fs, and gene2fs on the request list. Plus the stuff used
> out of pending.
>
>>> Android's -t (which conflicts with ubuntu's -t ttylist) uses /proc/task
>>> to fetch stat and attr/current, blanks cmdline, and hardwires ppid to
>>> the thread parent (which it already seems to be in stat). So not a big
>>> change.
>>
>> the name of the option doesn't matter much. there's one hard-coded
>> callsite that i'll have to change when we switch ps and top anyway (a
>> bugreport includes both by default).
>>
>> i don't think there's any ppid hard-wiring. the top-level ps loop in
>> Android relies on the /proc weirdness you mention later: it iterates
>> through the visible /proc/\d+ entries, outputs the line for that pid,
>> then looks in its tasks subdirectory and calls the "show a line of
>> info about pid x" function for each tid in there.
>>
>>> Ubuntu's man page says says threads are ps -eLf. So -e is -A, -L is
>>> "show threads", and -f is full format and adds LWP and NLWP columns when
>>> used with -L. (And it DOESN'T seem to document that -L adds LWP
>>
>> here are the GNU options i alluded to:
>>
>> ~$ ps --help threads
>>
>> Usage:
>>  ps [options]
>>
>> Show threads:
>>   H                   as if they where processes
>
> where? (Typo for were? What?)
>
> Hmmm... according to the man page, that's process hierarchy (I.E.
> forest): group by parent/child and indent the cmdline.

(i'll reply to the rest later, but...)

i think you were so distracted by the typo that you missed that this
is "H", not "-H" :-)

try "ps auxH".

> According to ls /proc/*/task pid 9922 is a chromium fiddly bit that has
> multiple threads (9923 to 9940... Huh, I wonder if I can use presence of
> child threads to work around the darn chromium command line truncation
> bug in Ubuntu 14.04? http://landley.net/notes-2016.html#29-03-2016 .)
>
> anyway, ps -H shows pid 9922 but nothing under it, and does NOT show pid
> 9923.
>
>>  -L                   possibly with LWP and NLWP columns
>
> ps -LH gives me a usage message.
>
>>  -m, m                after processes
>
> There are two instances of -m, both lower case? (So... -mm means
> something different than -m?)

again, they're saying that "-m" works, as does "m".

> $ ps -m
>   PID TTY          TIME CMD
> 19869 pts/121  00:00:00 bash
>     - -        00:00:00 -
> 22293 pts/121  00:00:00 ps
>     - -        00:00:00 -
>
> Well that's nice...?
>
> $ ps -mH
>
> 290 lines of output, the header line of which is
> PID TTY      MAJFLT MINFLT   TRS   DRS  SIZE  SWAP   RSS  SHRD   LIB
> DT COMMAND
>
> And yes it's longer than 80 chars.
>
>>  -T                   possibly with SPID column
>
> ps -TH shows the two processes attached to this terminal (bash and ps
> -T), ps -TH does the same, and ps -ATH gives me a usage message...
>
> $ ps -Tm
>   PID  SPID TTY          TIME CMD
> 19869     - pts/121  00:00:00 bash
>     - 19869 -        00:00:00 -
> 22425     - pts/121  00:00:00 ps
>     - 22425 -        00:00:00 -
>
> This is just STUNNINGLY well designed, isn't it?
>
> Right, it looks like -T to show threads is the least _ILLOGICAL_ option,
> and then the
>
>>> Android's top is showing thread count as field #5, and "PCY" (scheduling
>>> policy) as field #8. And that's fetched with infrastructure that doesn't
>>> exist on Ubuntu: get_sched_policy() and get_sched_policy_name(). Um,
>>> does this need to be fixed up too? (My top has "pr" and "ni" in the
>>> default output. You can add SCHED with -o, but it's numeric, and the
>>> help text doesn't match the android fields (audio?).)
>>
>> yeah, don't worry about that stuff --- i'll send you patches for those
>> things later. (it's more cutils cruft. it's hairy enough and changes
>> from release to release that i think it would be a mistake for toybox
>> to try to do it itself.)
>
> Works for me. Lemme see what I can do with ps...
>
> Rob



-- 
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Android native code/tools questions? Mail me/drop by/add me as a reviewer.