From rob at landley.net  Sat Feb  4 17:27:47 2017
From: rob at landley.net (Rob Landley)
Date: Sat, 4 Feb 2017 11:27:47 -0600
Subject: [Aboriginal] [Toybox] 3efd6d9512ac8c9370d62fcaa08d31c28bba1810
 broke ps without a tty
In-Reply-To: <CAJgzZoqE8+9427ZuLO3eDUyef15_6xv0Eq8pcPYwjQEnGC=27A@mail.gmail.com>
References: <CAJgzZoqE8+9427ZuLO3eDUyef15_6xv0Eq8pcPYwjQEnGC=27A@mail.gmail.com>
Message-ID: <e79020b8-9f3f-b8ea-67a3-99c055a4276c@landley.net>

On 02/03/2017 07:04 PM, enh wrote:
> (i've been trying to find time to look at this for days, but it's time
> to admit to myself that i won't and report what i've found...)

I actually found that yesterday but haven't checked in a fix yet.

The main holdup on cutting the toybox release has been that aboriginal
linux hasn't been a functional test bed to find obvious regressions like
this for a while, so I've been banging on
https://github.com/richfelker/musl-cross-make to get toolchains out of
that (using the attached build script), then banging on
https://github.com/landley/mkroot to get it up to speed as a root
filesystem builder, and now I'm trying to figure out what the kernel
build script part should look like. (So far I still need 2/3 of the old
sources/targets from aboriginal, although maybe I can do it as an
if/else staircase in one script.)

And I got far enough to notice that ps was nuts on a serial console,
because setting the default width to 99999 _without_ setting FLAG_w
cause it to try to _pad_ each line out to 99999 with spaces.

Oops.

If the size probe fails I can set the default width to 80 instead of
99999, or I can set FLAG_w. Not sure which is better yet. (With 80 the
columns would at least line up,and you can always export COLUMNS=123 if
you need a specific different size.

In theory the ANSI size probe should go in here somewhere. In practice
we're not already reading data from stdin so parsing the results isn't a
natural part of the program flow.

(I note that top is also glitched without a tty. I suspect receipt of
the ansi probe causes a redraw... which sends out a new ansi probe.)

Moral of the story: don't let your main test environment fall too far
behind or stupid stuff accumulates. I'm working on it...

Rob

P.S. I've spent hours trying to come up with a proper news entry for
aboriginal that covers why I ended the project and what I'm replacing it
with, but
http://lists.landley.net/pipermail/aboriginal-landley.net/2017-January/002594.html
is the closest I've come up with so far.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: all2.sh
Type: application/x-shellscript
Size: 1894 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20170204/30beb4b6/attachment.bin>

From enh at google.com  Sat Feb  4 19:03:08 2017
From: enh at google.com (enh)
Date: Sat, 4 Feb 2017 11:03:08 -0800
Subject: [Aboriginal] [Toybox] 3efd6d9512ac8c9370d62fcaa08d31c28bba1810
 broke ps without a tty
In-Reply-To: <e79020b8-9f3f-b8ea-67a3-99c055a4276c@landley.net>
References: <CAJgzZoqE8+9427ZuLO3eDUyef15_6xv0Eq8pcPYwjQEnGC=27A@mail.gmail.com>
	<e79020b8-9f3f-b8ea-67a3-99c055a4276c@landley.net>
Message-ID: <CAJgzZopPbyD__YB8rzXpnApHyK0Uqv7eAcwkxL5yAL-zC93yRA@mail.gmail.com>

On Sat, Feb 4, 2017 at 9:27 AM, Rob Landley <rob at landley.net> wrote:
> On 02/03/2017 07:04 PM, enh wrote:
>> (i've been trying to find time to look at this for days, but it's time
>> to admit to myself that i won't and report what i've found...)
>
> I actually found that yesterday but haven't checked in a fix yet.
>
> The main holdup on cutting the toybox release has been that aboriginal
> linux hasn't been a functional test bed to find obvious regressions like
> this for a while, so I've been banging on
> https://github.com/richfelker/musl-cross-make to get toolchains out of
> that (using the attached build script), then banging on
> https://github.com/landley/mkroot to get it up to speed as a root
> filesystem builder, and now I'm trying to figure out what the kernel
> build script part should look like. (So far I still need 2/3 of the old
> sources/targets from aboriginal, although maybe I can do it as an
> if/else staircase in one script.)
>
> And I got far enough to notice that ps was nuts on a serial console,
> because setting the default width to 99999 _without_ setting FLAG_w
> cause it to try to _pad_ each line out to 99999 with spaces.
>
> Oops.
>
> If the size probe fails I can set the default width to 80 instead of
> 99999, or I can set FLAG_w. Not sure which is better yet. (With 80 the
> columns would at least line up,and you can always export COLUMNS=123 if
> you need a specific different size.

without a tty is the default for Android, where most folks are using
"adb shell ..." or moral equivalent. (similar to running everything
via ssh.) so you can't really tell the difference between "i'm a
programmatic use" and "i'm a human hiding behind something like ssh".

-w would certainly be less disruptive for Android use cases (because
it won't break scripts; traditional Android ps _always_ overflowed).

> In theory the ANSI size probe should go in here somewhere. In practice
> we're not already reading data from stdin so parsing the results isn't a
> natural part of the program flow.
>
> (I note that top is also glitched without a tty. I suspect receipt of
> the ansi probe causes a redraw... which sends out a new ansi probe.)

quitting top if it ran like this is weird too. the escape sequences to
reset the terminal don't make it. i've never mentioned it because i
don't have any suggestion for how we could do better.

> Moral of the story: don't let your main test environment fall too far
> behind or stupid stuff accumulates. I'm working on it...
>
> Rob
>
> P.S. I've spent hours trying to come up with a proper news entry for
> aboriginal that covers why I ended the project and what I'm replacing it
> with, but
> http://lists.landley.net/pipermail/aboriginal-landley.net/2017-January/002594.html
> is the closest I've come up with so far.



-- 
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Android native code/tools questions? Mail me/drop by/add me as a reviewer.


From rob at landley.net  Mon Feb  6 23:34:23 2017
From: rob at landley.net (Rob Landley)
Date: Mon, 6 Feb 2017 17:34:23 -0600
Subject: [Aboriginal] Replace ADDR with BIT in ps -l?
In-Reply-To: <CAJgzZopPbyD__YB8rzXpnApHyK0Uqv7eAcwkxL5yAL-zC93yRA@mail.gmail.com>
References: <CAJgzZoqE8+9427ZuLO3eDUyef15_6xv0Eq8pcPYwjQEnGC=27A@mail.gmail.com>
	<e79020b8-9f3f-b8ea-67a3-99c055a4276c@landley.net>
	<CAJgzZopPbyD__YB8rzXpnApHyK0Uqv7eAcwkxL5yAL-zC93yRA@mail.gmail.com>
Message-ID: <5f487d45-5e61-8dd7-86dc-057aa95b4785@landley.net>

On 02/04/2017 01:03 PM, enh wrote:
> without a tty is the default for Android, where most folks are using
> "adb shell ..." or moral equivalent. (similar to running everything
> via ssh.) so you can't really tell the difference between "i'm a
> programmatic use" and "i'm a human hiding behind something like ssh".
> 
> -w would certainly be less disruptive for Android use cases (because
> it won't break scripts; traditional Android ps _always_ overflowed).

Ok, that's in, and I think I've fixed all the weirdness (and top isn't
re-querying endlessly, although I need to change top not to do a screen
clear but instead to [K kill each line before redrawing it, so the
display doesn't flicker so badly when drawn via serial).

But, one more issue: on an 80 char terminal "ps -Al" leaves _four_
characters for CMD. And when the last one's + that means there's
actually 3 it's displaying.

The main culprit is "ADDR", which needs lots more space to display real
data on 64 bit systems. But it's not usually very useful (wchan is more
useful) and only does this for your own processes unless run as root.

Unfortunately removing it would break scripts that count fields with awk
or similar, so

So I suggest replacing ADDR with BIT, which leaves 13 chars for CMD?
(Much more readable.) We could also trim one space from TTY (at least on
my system, I haven't seen pts/123 go to 4 digits yet so there's always a
blank space).

Elliott: would this break obvious android stuff?

Rob


From enh at google.com  Tue Feb  7 00:46:46 2017
From: enh at google.com (enh)
Date: Mon, 6 Feb 2017 16:46:46 -0800
Subject: [Aboriginal] Replace ADDR with BIT in ps -l?
In-Reply-To: <5f487d45-5e61-8dd7-86dc-057aa95b4785@landley.net>
References: <CAJgzZoqE8+9427ZuLO3eDUyef15_6xv0Eq8pcPYwjQEnGC=27A@mail.gmail.com>
	<e79020b8-9f3f-b8ea-67a3-99c055a4276c@landley.net>
	<CAJgzZopPbyD__YB8rzXpnApHyK0Uqv7eAcwkxL5yAL-zC93yRA@mail.gmail.com>
	<5f487d45-5e61-8dd7-86dc-057aa95b4785@landley.net>
Message-ID: <CAJgzZopXYCbXcjD4M0ZBEzOkb1OpHiEfLP5S5wSEpboNh+hG=g@mail.gmail.com>

On Mon, Feb 6, 2017 at 3:34 PM, Rob Landley <rob at landley.net> wrote:
> On 02/04/2017 01:03 PM, enh wrote:
>> without a tty is the default for Android, where most folks are using
>> "adb shell ..." or moral equivalent. (similar to running everything
>> via ssh.) so you can't really tell the difference between "i'm a
>> programmatic use" and "i'm a human hiding behind something like ssh".
>>
>> -w would certainly be less disruptive for Android use cases (because
>> it won't break scripts; traditional Android ps _always_ overflowed).
>
> Ok, that's in, and I think I've fixed all the weirdness (and top isn't
> re-querying endlessly, although I need to change top not to do a screen
> clear but instead to [K kill each line before redrawing it, so the
> display doesn't flicker so badly when drawn via serial).
>
> But, one more issue: on an 80 char terminal "ps -Al" leaves _four_
> characters for CMD. And when the last one's + that means there's
> actually 3 it's displaying.
>
> The main culprit is "ADDR", which needs lots more space to display real
> data on 64 bit systems. But it's not usually very useful (wchan is more
> useful) and only does this for your own processes unless run as root.
>
> Unfortunately removing it would break scripts that count fields with awk
> or similar, so
>
> So I suggest replacing ADDR with BIT, which leaves 13 chars for CMD?
> (Much more readable.) We could also trim one space from TTY (at least on
> my system, I haven't seen pts/123 go to 4 digits yet so there's always a
> blank space).
>
> Elliott: would this break obvious android stuff?

it's hard to be 100% sure (because of ps' weird command-line parsing),
but afaics from internal code search, no scripts or code use ps' "l"
option.

sounds fine to me, and i'll deal with anything should it break.

> Rob



-- 
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Android native code/tools questions? Mail me/drop by/add me as a reviewer.


From rob at landley.net  Sat Feb 18 10:29:36 2017
From: rob at landley.net (Rob Landley)
Date: Sat, 18 Feb 2017 04:29:36 -0600
Subject: [Aboriginal] Latest cross-compiler-x86_64 stopped working as
 /usr/bin/x86-64-gcc symlink
In-Reply-To: <b884f073-9b38-14b2-a207-b32e0c819b07@landley.net>
References: <CAK1hOcPposMSDbh5U6db6JRxp1c5a4iMLx7BHwNUAo8VZB4f0A@mail.gmail.com>
	<b884f073-9b38-14b2-a207-b32e0c819b07@landley.net>
Message-ID: <bd1212d9-6a7e-cf21-b392-cddc8c63d71e@landley.net>

On 01/30/2017 09:13 PM, Rob Landley wrote:
> On 01/30/2017 03:48 PM, Denys Vlasenko wrote:
>> Hi Rob,
>>
>> Needing to test some musl glob() issues, today I downloaded
>>
>> http://landley.net/aboriginal/downloads/binaries/cross-compiler-x86_64.tar.gz
> 
> Timing. [1]
> 
> Aboriginal Linux is mothballed, and is being replaced by two projects:
> 
>   toolchain: https://github.com/musl-cross-make [2]
>   filesystem: https://github.com/landley/mkroot

Attached is an updated musl-cross-make build script that produces all
the cross and native compilers I've managed to get working so far.

It's basically git clone the repo, cd into it, run the script, leave it
going overnight, look in the "output" directory.

I've built mkroot filesystems with all those toolchains, and the ones I
tried worked. (Except armv8l actually seems to be armv7l, and armv7r is
nommu and needs to be pie like armv7m because arm fdpic hasn't been
merged upstream yet.)

It doesn't have all the architectures the old stuff did (sparc is most
conspicuously absent, but ppc64, s390x, and microblaze are new so...)

> And I've giving a tutorial based on the new stuff at ELC next month:
> 
> https://openiotelcna2017.sched.com/event/9Ith/tutorial-building-the-simplest-possible-linux-system-rob-landley-se-instrumentscom

Which is this coming tuesday. No idea when/if video goes up but I should
have the tutorial materials up on the j-core website before too much longer.

Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mcm-buildall.sh
Type: application/x-shellscript
Size: 2066 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20170218/67d047da/attachment.bin>

From rob at landley.net  Sun Feb 19 01:30:45 2017
From: rob at landley.net (Rob Landley)
Date: Sat, 18 Feb 2017 19:30:45 -0600
Subject: [Aboriginal] [Toybox] 3efd6d9512ac8c9370d62fcaa08d31c28bba1810
 broke ps without a tty
In-Reply-To: <CAJgzZopPbyD__YB8rzXpnApHyK0Uqv7eAcwkxL5yAL-zC93yRA@mail.gmail.com>
References: <CAJgzZoqE8+9427ZuLO3eDUyef15_6xv0Eq8pcPYwjQEnGC=27A@mail.gmail.com>
	<e79020b8-9f3f-b8ea-67a3-99c055a4276c@landley.net>
	<CAJgzZopPbyD__YB8rzXpnApHyK0Uqv7eAcwkxL5yAL-zC93yRA@mail.gmail.com>
Message-ID: <058f0c2a-2356-b0eb-da5e-2ce8db65c66c@landley.net>

On 02/04/2017 01:03 PM, enh wrote:
>> In theory the ANSI size probe should go in here somewhere. In practice
>> we're not already reading data from stdin so parsing the results isn't a
>> natural part of the program flow.
>>
>> (I note that top is also glitched without a tty. I suspect receipt of
>> the ansi probe causes a redraw... which sends out a new ansi probe.)
> 
> quitting top if it ran like this is weird too. the escape sequences to
> reset the terminal don't make it.

They're making it here...?

Define "quit". Is adb exiting before the output is flushed? Both Q and
ctrl-C are interpreted as key sequences, not signals (because stdin is
in raw mode), so the cleanup shouldn't get interrupted...?

> i've never mentioned it because i
> don't have any suggestion for how we could do better.

I'd like to fix it but am not reproducing the problem here.

>> Moral of the story: don't let your main test environment fall too far
>> behind or stupid stuff accumulates. I'm working on it...

Sounds like I'll need a proper android test environment soon.

Rob


From enh at google.com  Mon Feb 20 18:23:23 2017
From: enh at google.com (enh)
Date: Mon, 20 Feb 2017 10:23:23 -0800
Subject: [Aboriginal] [Toybox] 3efd6d9512ac8c9370d62fcaa08d31c28bba1810
 broke ps without a tty
In-Reply-To: <058f0c2a-2356-b0eb-da5e-2ce8db65c66c@landley.net>
References: <CAJgzZoqE8+9427ZuLO3eDUyef15_6xv0Eq8pcPYwjQEnGC=27A@mail.gmail.com>
	<e79020b8-9f3f-b8ea-67a3-99c055a4276c@landley.net>
	<CAJgzZopPbyD__YB8rzXpnApHyK0Uqv7eAcwkxL5yAL-zC93yRA@mail.gmail.com>
	<058f0c2a-2356-b0eb-da5e-2ce8db65c66c@landley.net>
Message-ID: <CAJgzZoqfUhoXywT4s3UB0_mix-7tWQb=OoQ9wqvAqU8aMSSvqw@mail.gmail.com>

On Sat, Feb 18, 2017 at 5:30 PM, Rob Landley <rob at landley.net> wrote:
> On 02/04/2017 01:03 PM, enh wrote:
>>> In theory the ANSI size probe should go in here somewhere. In practice
>>> we're not already reading data from stdin so parsing the results isn't a
>>> natural part of the program flow.
>>>
>>> (I note that top is also glitched without a tty. I suspect receipt of
>>> the ansi probe causes a redraw... which sends out a new ansi probe.)
>>
>> quitting top if it ran like this is weird too. the escape sequences to
>> reset the terminal don't make it.
>
> They're making it here...?
>
> Define "quit". Is adb exiting before the output is flushed? Both Q and
> ctrl-C are interpreted as key sequences, not signals (because stdin is
> in raw mode), so the cleanup shouldn't get interrupted...?

if you "adb shell" and then run "top" interactively, both q and ctrl-c
work fine.

if you "adb shell top", you need "q\n" to politely quit, and ctrl-c kills adb.

i think this is intended behavior (and you can use -t/-T to control
it, as with ssh(1)), though i'm increasingly uncertain that this was
really a good model to emulate, given that i have no data about how
many people use "interactive" versus "non-interactive" and a strong
suspicion that people run interactive commands like top in what ssh
would (and modern adb does) interpret as non-interactive mode.

>> i've never mentioned it because i
>> don't have any suggestion for how we could do better.
>
> I'd like to fix it but am not reproducing the problem here.

as i was trying to say, i don't think it's your bug. i think it's an
unfortunate interaction (no pun intended) between adb and top's ideas
of interactivity. i think just i'm trying to psych myself up to saying
adb shouldn't match ssh in this...

(ssh+top sidesteps this by ssh not passing through $TERM when
non-interactive and top refusing to start without $TERM.)

>>> Moral of the story: don't let your main test environment fall too far
>>> behind or stupid stuff accumulates. I'm working on it...
>
> Sounds like I'll need a proper android test environment soon.
>
> Rob



-- 
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Android native code/tools questions? Mail me/drop by/add me as a reviewer.


From rob at landley.net  Mon Feb 20 19:41:59 2017
From: rob at landley.net (Rob Landley)
Date: Mon, 20 Feb 2017 13:41:59 -0600
Subject: [Aboriginal] ELC tutorial on mkroot.
Message-ID: <38e31a7c-6274-ab7d-f4df-5ae51919a807@landley.net>

I'm doing a 2-hour tutorial at the Embedded Linux Conference in Portland
tomorrow, on the successor project to Aboriginal Linux (mkroot).

https://openiotelcna2017.sched.com/event/9Ith/tutorial-building-the-simplest-possible-linux-system-rob-landley-se-instrumentscom

Just FYI, if you already happen to be there. (In theory they'll record
it and put video online.)

Rob