[Toybox] [PATCH] scan_key: support more terminals.

[enh]

On Sat, Mar 23, 2019 at 4:11 PM Rob Landley <rob at landley.net> wrote:
>
> On 3/23/19 2:44 PM, enh via Toybox wrote:
> > Although we can get away with ignoring termcap/terminfo on the output
> > side by restricting ourselves to generally-supported escape sequences,
>
> There's documentation on this, by the way:
>
> http://man7.org/linux/man-pages/man4/console_codes.4.html

well, personally i always use

  https://invisible-island.net/xterm/ctlseqs/ctlseqs.html

i'm not sure i've ever done anything more than hit ctrl-alt-del at an
actual linux console :-)

> I have a note on my todo list to look at this one especially:
>
>       ESC [ ? 1 h
>               DECCKM (default off): When set, the cursor keys send an ESC O
>               prefix, rather than ESC [.
>
>   $ echo -e '\e[?1h'
>   $ sleep 10
>   ^[OA^[OA^[OA^C
>
> I probably want to add '\e[?1l' to my setup code...

there's no one true place for this at the moment though, right? the
*cleanup* has been factored out, but the setup not.

> > the input side is trickier because we need to support the sequences sent
> > by common terminals. Luckily, this isn't is as bad as it sounds because
> > only Home/End commonly differ. But it does mean we need a slightly
> > different implementation to deal with the many-to-one mapping.
>
> What _were_ the other sequences for home/end? Looks like it was just \eO instead
> of \e[ prefixes?

see https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h2-PC-Style-Function-Keys
and following sections for a good roundup of all the weirdness that's
out there.

i commented the various things i hit (though i only gave one example
source for each):

+  // VT102/VT220 escapes.
+  {KEY_HOME, "\033[1~"},
+  {KEY_INSERT, "\033[2~"},
+  {KEY_DELETE, "\033[3~"},
+  {KEY_END, "\033[4~"},
+  {KEY_PGUP, "\033[5~"},
+  {KEY_PGDN, "\033[6~"},

+  // "Normal" "PC" escapes (xterm).
+  {KEY_HOME, "\033OH"},
+  {KEY_END, "\033OF"},

+  // "Application" "PC" escapes (gnome-terminal).
+  {KEY_HOME, "\033[H"},
+  {KEY_END, "\033[F"},


> > Since we can't use TAGGED_ARRAY for this (without inflicting pain on all
> > the callers)
>
> Or adding a redirect field that says "this is actually this other one".
>
> Long ago I pondered a "list of strings that can generate this key" but the hard
> part is that the numeric semicolon syntax is arbitrarily extensible (just like
> with color changes the other way, ala ls.c:
>
>     if (flags & FLAG_color) {
>       color = color_from_mode(st->st_mode);
>       if (color) printf("\033[%d;%dm", color>>8, color&255);
>     }
>
> Most _mostly_ it's just pairs that seem to be reported? (But not always. I think
> I blogged about this years and years ago, back when I was working out how to fix
>
>   http://lists.busybox.net/pipermail/busybox/2008-October/067397.html
>
> And the FUN part is that function keys are reporting... inconsistent weirdness.
> Shift F1 in the terminal is \e[25~ and shift F2 is 26, but then it goes 28 29 31
> 32 33 34 (skipping 27 and 30 but _not_ 33), and then refuses to report shift F9
> or higher at all. (Wheee!)

see the ctlseqs link. for function keys i don't think there's actually
a sensible option. they're all weird. (i'd say that the VT220
sequences are relatively sane in general.)

> Oh, and here's the ctrl-shift function key sequence in an xfce terminal, F1
> through F12:
>
> ^[O1;6P^[O1;6Q^[O1;6R^[O1;6S^[[15;6~^[[17;6~^[[18;6~^[[19;6~^[[20;6~^[[23;6~^[[24;6~
>
> (It decided that ctrl-shift-F10 should pop up a menu so I didn't get that one,
> but that's plenty crazy right there. First 4 start with the non-current escape
> sequence, which at least lets me distinguish the R response from a cursor probe
> response. Then it's more like the Linux text mode one except I don't get
> _anything_ for ctrl-shift-function keys in there...)
>
> > I've also switched to OR-ing in the modifier keys, so we
> > have (say) KEY_UP|KEY_SHIFT rather than a separate KEY_SUP. This also
> > generalizes better should we ever need to support multiple modifiers at
> > once.
>
> I've pondered doing something like that and waited for a user to show up. :)

so what i'm actually doing here is i had a day off and needed a hex
editor for the thing i was dicking about with, and made some changes
to hexedit to make it a bit more usable. one of those things was to
make home/end and ctrl-home/ctrl-end more like other editors. i split
off this patch to tty.c because there are so many ways of doing this i
thought we'd deal with one bikeshed at a time. but the downside is
that it robs the patch of some of its motivation. i am actually trying
to use the ctrl-home/ctrl-end sequences.

> Especially since such a user might be able to do a "survey of available inputs",
> although now that terminal programs are less of a thing that's probably mostly
> "ssh from mac, bsd, and putty". (In theory the various terminal window programs
> for gnome/kde/xfce are also a varying source of inputs, but they all seem to
> _mostly_ copy linux text mode?)
>
> The real problem is to _get_ a survey of all the keys I have to boot into an
> xterm with no window manager, because otherwise half of the key chords are
> intercepted. (Did you know that ctrl-alt-cursor key switches desktops in the
> desktop switcher?) Which means I have to install a bunch of terminal programs,
> rip them out of their desktop context and boot them directly under /usr/bin/X,
> try all the keys and record the results, and hopefully work out patterns.

i don't think this is as hard as you think: the modifiers are the same
for any key, so as long as you can test with _some_ key, you know the
modifiers are the same for the rest.

you also don't need _no_ window manager. you can use a window manager
that doesn't touch the keys:
https://github.com/software-jessies-org/jessies/tree/master/lwm and
then -- if you need it -- use a separate program to run commands in
response to interesting keystrokes:
https://github.com/software-jessies-org/jessies/tree/master/x11-extras

(disclaimer: there are at least two forks of lwm because i haven't
touched it in 20+ years.)

> And yes I asked Rich Felker, who wrote a terminal program of his own long ago
> (http://git.musl-libc.org/cgit/uuterm/) if he had useful resources, and he
> didn't. :(

if you want to know the legit escape sequences, see ctlseqs. yes, it's
hideously complicated, but it accurately describes reality :-(

if you want to know about the underlying keystrokes coming from X11 or
whatever, for the Java terminal we wrote
(https://github.com/software-jessies-org/jessies/wiki/Terminator)
there's actually a thing in the Help > Debugging Tools menu that lets
you see the keycodes its getting (which are X11-style). you probably
have xev(1) installed too.

(there's also vttest, but that's mainly useful if you're trying to
emulate a terminal.)

> There's a lot of complexity in this that the current parser is unaware of, and I
> would love to improve it it, but I'd probaby want to _start_ with not having
> esc[ repeated at the start of most sequences that could _also_ produce escO
> under the right circumstances. (Except above, collision between function key and
> position report, it _is_ meaninful in some circumstances! Grrr...)

if you want to do a perfect job, i think you really have to parse the
input. and that's hairy enough that i'm not sure you actually want to
do that. (i don't think even vim gets it 100% right all the time,
though it's been a while since i've had to hammer Esc to unconfuse
it.) iirc i had a hack in some tty-based code i wrote which would (a)
pull aside any terminal size reports but also (b) make the assumption
that any key sequence comes in a burst, so if you see ^[ then a bunch
of stuff and then a pause, that was a key sequence even if you didn't
understand it (likewise another ^[ means "start again, even if you
didn't understand what's in your buffer").

given that some terminals let you choose the set of sequences to send
_regardless_ of what sequences its seen that would control this, i
think the "perfect" reader has to accept all the possible input
sequences if it's never to be confused.

> > To reduce the number of #defines, I've also switched from KEY_F1,
> > KEY_F2, and so on to KEY_FN+1, KEY_FN+2, and so on. This isn't obviously
> > necessary, and easily undone if we'd rather have move #defines in return
> > for slightly more natural naming.
>
> I'd like to work out what the design should _be_. And it's stayed on the todo
> list this long because that's nontrivial.

luckily i suspect in 2019 we mainly want function keys just so we can
throw them away and get on to the next meaningful keypress! but i can
easily put this back to one constant per key if you prefer.

> > To enable all this, I've inverted scan_key and scan_key_getsize so that
> > scan_key_getsize is now the underlying function, and we don't waste all
> > the top bits encoding width and height between scan_key and
> > scan_key_getsize.
>
> As long as we don't _send_ the probe when we don't mean to, life is good.
>
> > Tested by pressing Home and End in hexedit in all of the terminals
> > available to me.
>
> For regression testing, can I get a list of what those terminals _are_?

Terminator, xterm, gnome-terminal, xcfe-terminal, rxvt, and a special
guest appearance from macOS Terminal. (i didn't test iTerm2 or PuTTY
which are the big names missing, because i don't have a Mac or Windows
box i can install stuff on.)

i've just realized that to some extent i probably care about the
Windows "cmd" stuff too, given that i have users who'll be arriving
via adb.exe run on Windows! i've yet to hear any complaints for them,
so i'll quietly re-insert my head into the sand... (but add that to
the list of reasons why i think the dream of "one true sequence per
key" is futile...)

> Rob