[Toybox] ps and top (and Android)

[Rob Landley]

On 04/20/2016 08:47 PM, enh wrote:
> re: https://github.com/landley/toybox/commit/529d5db29a32db565d2ee0aeac133e47ec045e12,
> note that we're not switched over because neither ps nor top have
> thread support. everything on Android is threaded except init, so
> people would notice :-)

I was tracking whether or not it was in pending, but you're right. My bad.

Yesterday I found out I'm going to Japan on Sunday, so the release I was
trying to prepare at the end of the month got bumped up. I have a 9 hour
layover in San Francisco on Sunday, and I'm hoping to upload it then.

I have like 6 bugs I need to fix between now and then (the find
segfault, the new bzcat segfault, expr memleak, etc). Plus I'm in the
process of cleaning up another half dozen commands out of pending, and I
should get at least a couple of them closed out and promoted this release.

And then there's the "build LFS with it" test, which I'm now trying to
migrate to LFS 7.8. Busy week. :)

> there are a bunch of little things too that i can send patches for,

Sure, throw 'em on the pile. :)

> but the thread support seems like a more architectural thing so i was
> waiting to see what you were going to do. (not least because GNU ps
> has three or four different modes for showing threads, their 'H' being
> closest to the Android -t.)

That's pretty much why I haven't done it yet: figuring out the right
thing to do is always the hard part, coding it's generally much easier.

I used threads quite extensively on OS/2, then saw pthreads and went
"ew" and never really got back into 'em. Given that unix can trivially
map shared memory and synchronize via pipes, I've just done that when I
needed to. (Java kinda forced threads down people's throats because they
refused to implement poll/select or nonblocking I/O for ages...)

So I'm not hugely familiar with what the output should look like for
threads. Let's see... what has multiple entries under /proc/*/task...
chromium of course. What the heck is "whoopsie"? (The ubuntu crash
reporter, which does not have a man page.)

Alright, it looks like all the threads show up in /proc but don't report
assocation to a tty, so get filtered out. (/proc/816/task/819 has a
subset of the same data in /proc/819, seems to be a strict subset? Do
ALL the threads exist at the top level?

  $ for i in /proc/*/task/*; do [ ! -d /proc/$(basename $i) ] && \
    echo $i; done

Yes, they all show up in /proc. So in theory all I have to do is adjust
the filtering parameters and possibly the display fields.

Android's -t (which conflicts with ubuntu's -t ttylist) uses /proc/task
to fetch stat and attr/current, blanks cmdline, and hardwires ppid to
the thread parent (which it already seems to be in stat). So not a big
change.

Ubuntu's man page says says threads are ps -eLf. So -e is -A, -L is
"show threads", and -f is full format and adds LWP and NLWP columns when
used with -L. (And it DOESN'T seem to document that -L adds LWP

LWP an alias for tid and NLWP is an alias for thcount (thread count). So
crazy gnu names for things that already have common names.

Just confirming: the posix ps page does not use the word "thread", and
there isn't a posix top page. Of course not, that would be far too useful...

So. If /proc/$PPID/tasks/$PID exists, then this is a thread: don't
display it. Except my ps -A isn't currently displaying 819, but when I
ls /proc/819 it's there...?

Oh you bastard. Whoever wrote proc is cheating, the synthetic filesystem
isn't showing the PID in the directory listing, but the thread is there
at the top level if you try to access it. So my test was invalid because
I expected SANITY from this filesystem. Grrr. (Dear Linux: your wires
are showing.)

Right, ok, I need to iterate through each pid's tasks subdirectory to
enumerate threads. I can do that. The top level view has several extra
files that task view doesn't, which is /proc being cute again: if I look
at /proc/816/tasks/819 I get a different set of files than /proc/819. It
looks like task view has a "children" that top level doesn't, and the
top level view adds autogroup, coredump_filter, map_files, mountstats,
task, and timers. None of which the current infrastructure is reading,
so yes I can just look at the tasks subdir with -L. Then have -L add TID
after PID, and have -f add TCNT.

Right, that's probably what ps should do. Now, what does TOP do? Rummage
rummage... H toggles thread view. It turns "Tasks:" into "Threads:" in
the header and makes the S column width 1 instead of width 2? Huh...

Android's top is showing thread count as field #5, and "PCY" (scheduling
policy) as field #8. And that's fetched with infrastructure that doesn't
exist on Ubuntu: get_sched_policy() and get_sched_policy_name(). Um,
does this need to be fixed up too? (My top has "pr" and "ni" in the
default output. You can add SCHED with -o, but it's numeric, and the
help text doesn't match the android fields (audio?).)

Next question: do I have to make pgrep and pkill thread aware? Let's
see... pkill doesn't care, pgrep has -w to show tid instead of pid.
Sounds straightforward enough...

Rob