[Toybox] [PATCH] Implement dmesg -T.

[Rob Landley]

On 04/04/2017 12:32 PM, enh wrote:
> On Mon, Apr 3, 2017 at 2:43 PM, Rob Landley <rob at landley.net
> <mailto:rob at landley.net>> wrote:
> 
>     On 04/03/2017 03:03 PM, enh wrote:
>     > since i have to maintain a parallel build system, promotion/demotion
>     > just mean extra work for me, so i'm always happier with the status quo :-)
> 
>     Trying to converge the build systems is on my todo list. :)
> 
>     (But today I'm playing with GPS signal tracking heuristics for $DAYJOB.
>     I'm finally seeing #%*(&%&# bit edges in the raw antenna data! Pretty
>     sure I have to correct code phase _before_ trying to track doppler
>     because it's just too noisy otherwise, but I might be able to use the
>     signal strength (vector length) as a weight for the phase angle deltas.
>     Dunno if that's helpful yet. This is _so_ much easier since I converted
>     everything to polar coordinates...)
> 
>     > as for the more recent patches i've been sending... it's not that i hate
>     > awk -- i'd actually love to have the time to sit down and write toybox
>     > awk -- but i'm trying to get to where we pass all the toybox tests [for
>     > the toys we're using] on Android.
> 
>     I note that some of the tests are broken and most are incomplete. (I
>     don't have a pending directory for test code, and large chunks of it
>     were contributed.)
> 
> that's all the more reason why i want to go through them. at the moment
> i think we're not paying much attention to them because we know they're
> not great. i'd like to get to the point where "all tests passing" is the
> default.

I have a local diff adding a second "testcmd" function (which differs
from testing in that it implies $C at the start of the command line, so
you don't have to repeat the command name. (See commit ee14fc396dff
adding that late last year: $CMDNAME is the name of the command being
tested, $C is an absolute path to that command to run so we bypass shell
builtins when we use it.)

Then I'm going through and converting the commands one by one to use it,
and using "I converted that to the new function" (using it at least
once) as a proxy for having reviewed the tests.

Unfortunately, actually reviewing the tests goes off on tangents very
easily. And the failure mode of having 15 minutes at a time to work on
toybox is I wind up with lots of partially finished things all over the
tree because I don't remember what I was working on last and keep trying
to grab a small todo item I might be able to finish in one sitting. (Or
at least get it to a point I can check in the next chunk before getting
pulled away. You never know...)

Sigh. I should at least check in the ones I've already gotten converted.
(Both of them. The third one alphabetically was blkid and I went off on
a tangent fixing something...)

The downside, of course, would be having yet more half-finished things
in the tree, and then new commands would start using cmdname _without_
having been fully vetted.

>     I keep meaning to do a "finishing pass" where I find commands I think
>     are done, read through the relevant specs (posix, lsb, and/or man page)
>     line by line, write a test for every statement and corner case I can
>     think of in its command.test file, then read through the command
>     implementation and writte any additional tests _that_ brings up (and
>     making sure the "deltas from posix" section is filled out and has
>     tests), make sure the command passes all of the tests, and then mark
>     both command and test READY (for 1.0).
> 
> yeah, that's what i've tried to do with gzip/gunzip/zcat too (though the
> weaker version in the absence of a spec of "verify every claim in the
> --help output").

Which is a start. :)

I need to fix the help output. Lemme fix ls -m first. (I checked in your
first text fixup! There's still a failing test. Legitimately, although
it's a whitespace issue.)

>     It's on the todo list. (It's part of the roadmap.html annotation.) Alas
>     I've been getting to spend about 4 hours on toybox in a _good_ week
>     recently. (Where "recently" is longer than I care to admit. And it tends
>     to be in 15 minute chunks, which are less useful than one 4 hour block
>     would be.)
> 
> that's one of the good things about at least polishing the existing
> tests: we'll need better tests eventually, but "something is better than
> nothing", and all the ones i've looked at so far were fittable into 15
> minute gaps between meetings.
> 
> the number of changes is starting to get hard to keep track of though,
> so if you could start merging some of them...

Indeed. I'll try to carve out some time this evening.

(When I hit a new issue like the ls -m thing I have the choice of
"follow the tangent and get distracted" or "throw it on the ever-growing
todo list containing items I haven't looked at in a year".)

My netbook finally crashed (failed to suspend properly, only power down
would fix it) so my 8 gazillion open windows/tabs with all the todo
items I hadn't properly written down on the todo lists got zapped. I
suppose that's a form of simplifying. Sigh.

> (and if there's something you want done differently, tell me now so i
> can start doing the others that way.)
>  
>     > ...and one reason i'm [finally] paying that the attention it deserves is
>     > that i have a libz-based gzip that i'll add to _tool_box in the short
>     > term and then work out how to get that into toybox (since you'll
>     > presumably want your no-dependencies version too, but you presumably
>     > won't want to reimplement the libz interface).
> 
>     I have a todo item to do libz wrapper versions of gzip/gunzip already,
>     but at this rate you may get to it before I do. :)
> 
> yeah, it's all done and passing the tests. (including the toybox tar
> tests, since tar shells out to gzip.) i'll get it committed to toolbox
> this week and then start worrying about porting to toybox.

And here I go "I should fix up and promote gzip decompression side, with
libz config option so you hae a model of what I want it to look like",
and that's ANOTHER tangent from ls -m and testcmd just from earlier this
email.

(And if I follow said tangent, that would leave this email window open
and half-replied to, that's another thing I lost maybe 3 dozen of in
last week's reboot although possibly thunderbird has drafts of them
auto-saved. Haven't checked yet, it's on the todo list...)

I need large blocks of time just to complete the giant pile of dangling
threads. Alas, I should be doing GPS code for $DAYJOB. (And I should be
checking in the rest of your patch list from yesterday, which
shortest-job-first says is probably the better scheduling decision...)

>     I was thinking of having the name "compress" be a front-end to deal with
>     multiple compression types (hence the help text, and things like
>     handling gzip zlib format with command line options), but that was a bit
>     of a cul-de-sac and I need to back it out again. (If I did it that way
>     I'd either bundle gzip/bzip/xz into compress.c, or have horrible
>     cross-command dependencies, or have the engines of those under lib.
>     Doesn't seem worth it...)
> 
> yeah, that's how things are at the moment, which is one reason i decided
> to do this in toolbox first. i guess it will be easiest if my patch adds
> a new pending/gzip.c and just #if 0s out those parts of compress.c and
> then let you decide how you want to implement the non-libz variant?

I was doing that, it turned out to be hard, trying to remember why. (I
think there's some shared bit-buffer infrastructure that I probably
should move to lib/ but then that tangented into "gzip decompression is
so much slower than zlib because they're dealing with 2 bits at a time
in the initial decision and why was I not doing that...")

Can of worms, going back to poke at ls -m now. (And then I need to get
back to the GPS code.)

Rob