[Toybox] [PATCH] Seperate 'userdel' from testing syntax

[Rob Landley]

On 02/20/2015 11:46 AM, enh wrote:
> On Wed, Feb 18, 2015 at 11:10 AM, Rob Landley <rob at landley.net> wrote:
>> You could also add a hook to scripts/runtest.sh replacing the "eval" on
>> line 86 with a function similar to the do_loudly function starting on
>> scripts/make.sh line 19.
>>
>> What does an adb remote invocation look like? Anything like testing via
>> ssh? I've vaguely pondered adding a "passwordless ssh to remote machine
>> and run the test there" thing so I could set up an aboriginal linux
>> powerpc instance, do a static cross compile build and run the tests
>> remotely inside the emulated instance with the host's test script
>> driving things.
> 
> so that's the first problem... you give adb more credit than it
> deserves. of its many misfeatures, pertinent ones here are: it won't
> work in a pipe,

I've wrapped stuff in pipe to pty translation before to make something
that really cared about having a controlling terminal think it had one.
I've also done ratelimiting for crap like qemu's powerpc serial
emulation where incoming data bigger than the serial buffer size gets
discarded.

> it won't return the exit status of the command it ran,

You'll notice the "test" command isn't checking exit status either,
which is why all the "|| echo yes" and such.

> and it will add a \r to every line of output.

| dos2unix

> i need to fix all of
> those, but none of them have become the most urgent fire yet, and no
> one's sent completely convincing patches. facebook wrote their own adb
> client (but it's an extended subset and doesn't support Windows [or
> the Mac?] so it's not really a replacement).

Lovely.

What's the adb transport, anyway? Toybox netcat has a "-f" mode that
connects to a file instead of stdin/stdout, which I've used as
half-assed microcom before. (I believe the help text describes how. A
todo item is making a _proper_ microcom alias for it, but that involves
writing stty first.)

(Alas the recent netcat patch to change the command line option parsing
broke server mode. I have a todo item to fix that before the release but
I've been fighting with my aboriginal linux test environment for a
couple weeks now and that's a blocker for doing a release of either
because the lfs-bootstrap with current toybox on a current kernel is my
main regression test.)

> anyway... something like this lets you run stuff like the pwd tests on
> a connected Android device. ($ANDROID_SERIAL is how you tell adb which
> Android device to talk to if there are several connected, without
> having to specify it on every command line. since you'd want to set
> that anyway before running the tests, that seemed reasonable.)
> 
> diff --git a/scripts/runtest.sh b/scripts/runtest.sh
> index 8da1089..c4244a8 100644
> --- a/scripts/runtest.sh
> +++ b/scripts/runtest.sh
> @@ -83,8 +83,16 @@ testing()
> 
>    echo -ne "$3" > expected
>    echo -ne "$4" > input
> -  echo -ne "$5" | eval "$2" > actual
> -  RETVAL=$?
> +  if [ -n "$ANDROID_SERIAL" ]
> +  then
> +    adb shell "echo -ne '$5' | eval '$2' > /data/local/tmp/actual ;
> echo $? > /data/local/tmp/retval"
> +    adb pull /data/local/tmp/actual .
> +    adb pull /data/local/tmp/retval .
> +    RETVAL=`cat retval`
> +  else
> +    echo -ne "$5" | eval "$2" > actual
> +    RETVAL=$?
> +  fi

Is there a way to hook that up to the emulator?

(I have two old phones with android I keep eyeing as test devices, but
both are their respective ancient vendor installs from before I
upgraded. My current Nexus 5 is t-mobile stock because I use it as a
phone. Although the recent upgrade to version 5.0.1 has made me
seriously reconsider that policy...)

>    cmp expected actual > /dev/null 2>&1
>    if [ $? -ne 0 ]
> 
> 
>> (Alas, a generic solution would also require a shared filesystem since
>> the test script is doing mkdir and such to set up temporary files to
>> test on. But I could probably botch something up...)
> 
> ...but, yeah, this is a problem. any test script that does some file
> system setup is a problem.

Did I mention that I have the first 1/3 or so of a 9p fileserver here,
and that when I sat next to a Samba developer at texas linuxfest last
year he told me that writing an smb server that _just_ served the newest
protocol version in "posix" mode was probably a simpler protocol than
9p2000.L? (He sent me links and everything.)

It's on the todo list. Also, Apple apparently wrote their own samba
replacement as part of their "great GPL purge":

http://www.osnews.com/story/24572/Apple_Ditches_SAMBA_in_Favour_of_Homegrown_Replacement

http://meta.ath0.com/2012/02/05/apples-great-gpl-purge/

> one possibility i've wondered about is factoring that out into a
> script of its own. this would have two advantages: 1. only one place
> to fix all this stuff for Android/ssh/whatever and 2. less boilerplate
> in each test script. if there was some known file system tree for
> commands to work on.

Factoring which out into its own script, setting up a shared filesystem?

The long-term solution is for me to get a proper shell on android, which
means making toysh work well enough to build toybox, aboriginal linux,
linux from scratch, and eventually the AOSP build.

But not this week...

>>> as i've said, Android doesn't have /etc/passwd (or /etc/group), and it
>>> doesn't have setpwent/getpwent/endpwent (or equivalents). you *can* do
>>> uid/gid or name lookups though, because they do make some degree of
>>> sense. (so id(1) works.) i've considered implementing getpwent so that
>>> it would cycle through the well known users, but we don't actually
>>> have an example of anything that would use it, and it's really not
>>> obvious that we'd be doing anyone any favors --- code calling getpwent
>>> that wants to run on Android needs to think long and hard about
>>> exactly what it means by that, and whether it makes any sense at all.
>>
>> This I'm still trying to wrap my head around. Most of the android
>> educational resources I find assume you're programming an app in java.
>>
>> Even if all your uid/gid pairs are dynamically allocated, can't you have
>> an app that requests and tracks uid/gid pairs, and assigns names for
>> them? Ok, they wouldn't be sequential (although the containers
>> infrastructure can remap them so they could _appear_ sequential if it
>> really mattered because somebody wanted to run a cifs fileserver or
>> something)...
> 
> no, there's no API for any of that. i doubt non-system apps even have
> the ability to read the relevant databases.

Takes some of the fun out of it.

This only comes up because I don't know how much traditional linux
context the AOSP build expects the build machine to have. If I can dig
out from under the current todo and $DAYJOB stuff, I need to try to
bootstrap the AOSP build natively under Aboriginal Linux (because it's
the simplest most stripped down build environment I've managed to get
working, so I should only ever have to _add_ stuff to it, and there
should never be a thing in it that another development environment
wouldn't have. If aboriginal has it, that's because you can't build a
self-bootstrapping system without it, so no missed implicit dependencies.)

But just working out "these packages build in this order" and then
triggering the builds one at a time is probably going to be a bit of a
job. (Let alone mapping the actual prerequisites. But hey, I've done it
before...)

>>> so going back to your "in pending ... because Android", i think you
>>> should just ignore us as far as adduser/userdel are concerned. i don't
>>> think there's anything useful you can do.
>>
>> I've heard that before. :)
>>
>> It still sounds fixable. Containers are awesome. Pity the crazy systemd
>> guys keep trying to hog the infrastructure over in legacy linux land,
>> but one big advantage android has is that the systemd being gpl keeps it
>> _off_ of android, so you don't have to get any of "devfsd-tng" on you
>> and can await the train wreck with popcorn...
> 
> i'm not saying the implementation won't ever change, but afaik no one
> is working on it.

Then me working on it wouldn't interfere with anybody. :)

(Although by the time I get the chance to, somebody else might be so I
need to try to stay up to date. Presumably there's some mailing list I
should be on somewhere...)

> your best hope is for someone to decide that there's
> a big enough security advantage to make it worth the disruption.

Oh I'm not trying to disrupt. I was suggesting creating a new thing off
to one side that could run builds, not changing the context your
existing stuff runs in.

If the system layer could reserve a UID/GID range that will never be
used by existing apk installs (a thousand of each should be plenty,
somewhere up in the sixty thousands maybe), then you could have a system
tool that creates a new LXC style container that maps those UID/GID
values down to the first 1000 (so inside the container you see UID 0 and
outside the container you see UID 60000 for the same resources), and the
tool runs these processes in a chroot (pivot_root actually) in which
there's a conventional /etc/password and /etc/groups for those however
many of those first 1000 values are actually in use by the container.

That container wouldn't need to have any _other_ UID/GID values
accessible (map the rest of the range to the same "nobody" value), so
you could have multiple users without the rest of the system
particularly having to care.

So the _option_ to add a process context with more traditional unix user
semantics isn't necessarily that intrusive a change.

That said, the bigger question is whether the AOSP build actually
_needs_ multiple users. If it doesn't (and any build that runs as a
non-root user shouldn't), then we can worry about it later.

Rob

 1424473939.0