[Toybox] [PATCH] microcom: move from e[x]it to [q]uit.

[enh]

On Sun, Nov 5, 2023 at 8:16 AM Rob Landley <rob at landley.net> wrote:
>
> On 11/3/23 09:44, enh wrote:
> > On Fri, Nov 3, 2023 at 7:22 AM Rob Landley <rob at landley.net> wrote:
> >>
> >> On 11/2/23 16:56, enh via Toybox wrote:
> >> > This frees up 'x' for xmodem transfers later.
> >>
> >> Q to quit makes sense, adding xmodem... do you have an xmodem protocol extension
> >> in mind that doesn't pad the last packet up to 128 bytes?
> >
> > that is annoying, yes, but since my only purpose is transferring
> > executables, i don't think it matters? (and even in the case where it
> > does, toybox already has a truncate(1).)
>
> A transfer protocol known to modify files is not ideal.

no, i meant you can truncate(1) the resulting destination file based
on what ls/stat told you was the exact size of the source file. (if it
matters, which seems most likely to be a problem for text files
anyway, and why not just send those as text in 2023?)

> >> What xmodem command
> >> are you going to run at the other end to consume the data and send the replies,
> >
> > i was going to write one. the xmodem spec sounds like it's only going
> > to be tens of lines.
>
> There's a spec? It's like 3 status bytes and 128 byte packets... (I was on
> bulletin boards when I was 12. There's a nonzero chance I implemented it back on
> the commodore 64, and even then it was inferior to punter.)

https://pauillac.inria.fr/~doligez/zmodem/ymodem.txt

> >> and how will the terminal program know to run it?
> >
> > i think for unix implementations of xmodem it's more common for the
> > user to do that part? the xmodem spec does talk about having sx send
> > "rx\r" and rx ignoring crap like that at the start, but that seems
> > gross enough to wait until someone actually _needs_ that.
>
> Busybox never bothered to implement the sx command, so I can't check there, but
> I would expect them to do that because "I am at a command prompt, send file"
> would just work. :)
>
> Except for the part where xmodem doesn't know what the filename is. I'm guessing
> it would write a.out by default?

lrzsz takes a [required] filename:
https://manpages.debian.org/testing/lrzsz/rx.1.en.html

> >> I still lean towards running a child command here. Heck, _vi_ has that (for some
> >> reason), I know because I keep hitting it accidentally.
> >
> > yeah, like i said earlier when we were talking about the progress
> > indicator --- "call out to sx(1)" is the obvious way to implement
> > xmodem, so replacing my custom "cat-alike with progress indicator"
> > with _actual_ cat(1) wouldn't be unreasonable (even if i'd be a little
> > sad to lose my progress indicator).
>
> I vaguely intend to move the new granular count.c progress indicator into lib,
> possibly removing -l from count and just having it always do that.
>
> A bar graph mode isn't hard to do, and there's a bargraph() in ps.c that could
> also get genericized and moved to lib...
>
> >> Whether we then add a
> >> zmodem command to toybox is a separate question...
> >
> > *z*modem is way more complicated (and still kind of a mess).
>
> How about ymodem? Ymodem was an attempt to fix the problems with xmodem: 1k
> packet size since we were no longer on 300 baud modems, block 0 includes the
> filename and size so it doesn't pad the last block, and 16-bit CRC instead of 8
> bit checksum. If you're going to do a file transfer protocol from the 1980s BBS
> world, ymodem is... much less bad.

see the doc above for this --- it seems like (even back in 1988)
people were confused about whether those were "really" ymodem or just
common xmodem extensions. afaik the idea is that you _try_ 1024 byte
blocks and 16-bit CRCs, but fall back if they don't work.

> Zmodem was the BBS-era's streaming serial protocol, meaning it doesn't insert
> latency spikes between each packet waiting for the far end to acknowledge, but
> instead constantly sends data until it runs out of data. (It also doesn't modify
> the files it's sending.)
>
> I also note that I've been meaning to write an rsync.c for toybox, and under the
> covers rsync is another member of this "send files across a serial connection"
> family. It's usually tunneled over ssh but it's just "bidirectional pipe". And
> like zmodem, it sends data to the far end until it runs out of data to send,
> asynchronously reading return results and appending updates to its stream as it
> processes them. So no latency spikes waiting for the other side to reply to what
> it said, except at the very end waiting for the far side to say it's done.
>
> I.E. the next logical step past zmodem is rsync. The next logical step BEFORE
> zmodem is ymodem.
>
> By the way, there are UI questions we're just kinda glossing over here. If you
> want "send these files to the far end" to be integrated... that may be a call to
> rsync.

no, not at all. don't get me wrong, i'm sure there are people who'd
rather have rsync, but we're in "everything is broken [or was always
too stupid to cope]" land here...

i swear i didn't go looking for this, but over the weekend i found
myself watching a youtube video about
https://github.com/gfoot/picoprom#picoprom---dip-eeprom-programmer-based-on-raspberry-pi-pico
--- even in 2023 (well, okay, 2021), xmodem is still the lowest common
denominator of serial transfers. that's what i claim rx/sx are useful
for, and that's also why i'd claim that rz/sz _aren't_ useful. because
anyone doing anything fancy is probably on scp/rsync, not zmodem :-)

> We haven't got a midnight commander variant that lets you select files
> and get a file list. The traditional unix way of doing this would call the
> various tools in a shell script, with the output of one being the input to (or
> command line arguments of) another.

the users i'm aware of are all just dealing with one file.

> I believe busybox microcom's expected usage is you can exit microcom without
> closing the connection, type "sz file.txt" at the command line, and then cursor
> up twice in your command history and hit enter to get microcom back. I switched
> the exit character from ctrl-X to ctrl-] because somebody might want to use
> emacs and ctrl-] is what telnet used back on the sun workstations in 1992.

(heh, i didn't realize that was you. but, yes, i appreciated the "this
is like telnet". though i'm reliably informed that the younger
generation hasn't heard of telnet.)

> The sharp edge is knowing sz should write to /dev/ttyACM3, which is why people
> made wrapper shell scripts, and the potential justification for "type child
> process command line and I'll run it with stdin/stdout going to the serial
> connection". There was also an environment variable solution at some point, I think?

(separately i did wonder whether microcom given no argument but
finding itself on a system with only one /dev/ttyUSB* should just
default to that?)

> But that's why they didn't historically build xmodem or similar into the
> terminal program, because if microcom can exit and let you do stuff and then
> resume, you're not limited to what they bothered to integrate into the program.

is it, or is it just that no-one wrote an sx for busybox? (because
*mini*com definitely has these.)

> You can start up a slip/ppp session or something if you need to. But if you
> start integrating the functionality _into_ the program, you've never got enough
> options and it goes fractal fast.

yeah, to be clear --- i have no particularly strong feelings here. i
think just having rx/sx is 90% of the value. and i suspect anyone who
doesn't work out base64 and text paste themselves is going to need to
copy & paste something from a doc anyway...

> >> > Also, if we're going to have a single line of help, switch to the more
> >> > common format for such prompts
> >>
> >> More common where? All the hits for grep '"[^"]*[[].[]][^"]*"' toys/*/*.c
> >> lib/*.c main.c are false positives finding array access between two strings. I
> >> don't think I've seen toybox use this anywhere else?
> >
> > oh, yeah, since my unzip(1) is c++ and uses Android's libziparchive i
> > only upstreamed the unzip _tests_ to toybox, didn't i?
> >
> > in terms of _toybox_ precedent, well, that's why i had the original
> > multi-line format :-) (toybox telnet uses multiple lines because
> > _busybox_ telnet does.) i can't think of any other interactive
> > prompting in toybox.
>
> I'm not against having it, I just want to know what the basis is. :)
>
> > but (as i just spoilered), unzip is about the only "unix" tool (in the
> > weaker sense of "something most unix users have come across at some
> > point" anyway)
>
> Because it started life in DOS. Abort/Retry/Ignore. From there it became the
> default windows archiver (because Windows a coat of paint on top of DOS until
> the W2K problem), and was adopted by linux via the open source info-zip
> implementation I first used under OS/2 in ~1992.
>
> > that has an interactive prompt (when it needs to
> > overwrite something),
>
> $ touch pluvial
> $ rm -i pluvial
> rm: remove regular empty file 'pluvial'?
> $ touch pluvial
> $ toybox rm -i pluvial
> rm pluvial (y/N):
>
> Not saying it's good, just saying it _is_.

(that's a yes/no, not a menu.)

> That's showing your options in
> parentheses, because the gnu/dammit one not showing your options seems less user
> friendly. (Toybox has terse output because I don't assume the reader speaks
> english, "name of command you just ran can plausibly be interpreted as a verb,
> name of file, keys you can hit in reponse". I _also_ try not to assume they have
> experience with the posix command line, because there's a first time for everybody.)
>
> This isn't conceptually the same as "[a] thingy [b] more [c] whatsis" but for a
> sufficiently LONG list, I personally find "a) option b) addendum c) walrus"
> makes it clear that the letter goes with which description, not "If you build it
> he will come. To have him build it for you, press [1]."
>
> > and it uses this style. and it's certainly more
> > readable than the "busybox telnet squished onto one line" style, and
> > the fact that the checked-in spacing between options wasn't even
> > correct seemed like a strong indicator to me that "this is not the way
> > to do it" :-)
>
> A typo indicates which style is preferable?

for me that's always been one of the key "code smells" --- if the
first time i try to use a new API/UI i get it wrong, i probably need
to rethink what i'm doing :-)

even if you just want to argue it was hard to see the mistake you'd
made, sure --- but that's probably telling you something about the
immediate grokkability of this prompt.

which i think is what i like about the "[q]uit" style --- it
highlights the most interesting bits and forces you to be mnemonic.
which, yes, is also a limitation, but if/when that limitation starts
to bite i'd argue you have way too many options in one menu. (i
present the *mini*com menu as evidence of this. super fancy
"curses"-style menu, but a complete nightmare because there's just way
too much stuff there, some of it seemingly duplicate.)

> I haven't cleaned up telnet because I use dropbear and netcat. Last time I
> touched it other than treewide API changes looks like commit 755e040916d9 in
> 2014 and the only prompt that changed was:
>
> -  if(TT.port <= 0 || TT.port > 65535)
> -    error_exit("PORT can be non-zero upto 65535.");
> +  if(TT.port <= 0 || TT.port > 65535) error_exit("bad PORT (1-65535)");

(yeah, you're right that the telnet ui is from the initial commit. but
_that_ matches the busybox ui.)

> >> Then again, grep '"[^"]* .)[^"]*"' toys/*/*.c lib/*.c main.c  isn't finding any
> >> other non-false-positive hits. If you have a strong opinion about this, I can
> >> switch, but I just want to be clear that either way this seems to be introducing
> >> a new rule into the project, and you want that rule to be [a] command rather
> >> than a) command.
> >
> > well, i'm actually suggesting "[c]ommand" rather than "c) command".
>
> Which with e[x]it raises the the issue of breaks between multi-word descriptions.

yeah, tbh i'm not sure why you hated the busybox telnet style of
multiple lines; it certainly avoids that :-)

> > but, yes, i think if you prefer "c) command" then we should actually
> > switch to "toybox --help" style where it's _usually_ one per line (and
> > there's a whole _tab_ for clear and consistent spacing) ...
> > until/unless you end up with lots of lots of options, at which point
> > things get a bit ascii-art.
>
> I wanted to collate them together on a line because it's an inefficient use of
> space otherwise, and scrolling stuff off the top of the screen seems impolite.
>
> We need _a_ prompt, and we've cleared the next line, so we might as well make
> use of that space we just reserved for ourselves.
>
> We could instead do the "snapshot cursor position and jump to a reserved line"
> except we don't reserve a line, and jumping to the top might not be noticed and
> jumping to the bottom means scrolling the screen which we wouldn't want to do
> multiple types if they typo (since you don't have an ESC key but instead treat
> any unknown key as ESC with another line of message.)
>
> At which point I start going "I started writing not-curses code in hexedit.c and
> then a second user in less.c and I was going to flesh it out properly in vi.c
> and have watch.c share it... and if microcom ever needed anything more
> complicated that was also a potential user... except I didn't get to write vi. I
> was too slow. And the people who did write the one we are now committed to use
> didn't know about my infrastructure plans and where else I wanted to reuse
> common code. Which is part of the reason I haven't started the shell command
> line editing stuff yet, because that was ALSO tied into this thing that's
> missing a limb now.
>
> Oh well...
>
> > (_that_ "toybox --help" style was actually my first thought for toybox
> > telnet too, if i'm honest, but either choice seemed like "innovation"
> > and i fell on the side of "same ui as busybox" being "less
> > innovative". which then naturally led to me doing the same for
> > microcom.)
>
> I haven't looked much at pending/telnet.c. I haven't _used_ telnet it forever, I
> use ssh or netcat. Does it also have user interface parts like this?>

_busybox_ telnet does, and toybox telnet looks like it's trying to match that.

"real" telnet does not. it has a bare "telnet> " prompt and you have
to ask for help. (plus no-one but us remembers telnet, so i'm not
convinced even the busybox one really serves as precedent; i only used
it as such because i literally couldn't think of anything else even
remotely similar. and *mini*com is the specific nightmare i'm running
from!)

> I used telnet fairly heavily in college on sun workstations, but only a few
> times since. Early on I hit a case where opening a network socket to a telnet
> server, running "tar" at the far end, and then feeding it a tarball wound up
> with corrupted files because it was treating some of the data as escape
> sequences, and rather than reverse engineer why I installed ssh...
>
> > my other other thought (which i also didn't pursue because i liked the
> > telnet precedent of the "Escape character ..." line) was outputting
> > something like
> >
> > toybox microcom
> > Press ctrl-] q to quit. Press ctrl-] ? for help.
> >
> > on startup.
>
> Which I would not expect users to remember 30 second after it's scrolled off,
> and which also adds in-band signaling to something that's otherwise transparent
> so you can do the exit-and-wrap thing. (That's why busybox's microcom also works
> like that.)
>
> I tend to have unix sensibilities, where "I'm gonna use this command in a shell
> script" is the default assumption.

sure, but is microcom useful in that context? are you using
stty/cat/sx there instead?

> A big chunk of your userbase is coming over
> from windows where everything is gui (no two commands ever connect together,
> that would be a license violation) and they expect something more like:
>
>   https://clip.cafe/who-framed-roger-rabbit-1988/laughing-pull-the-lever/t/1/
>
>   https://pinterest.com/pin/324822191849854340
>
> One community dials phone numbers and the other has a contact list. One
> navigates to a destination and the other has elevator buttons for every known
> destination. One has ingredients, the other has microwaveable dinners.
>
> Reality is usually somewhere in between. I have leanings, which I am willing to
> be talked out of, but I worry about an ever-growing pile of special cases, vs
> I've had the same hammer for 30 years and learned lots of different ways to use it.
>
> I readily admit your first experience being "here's some flour and water" vs
> "microwaveable ramen" is a big difference. As Yoda said, "Quicker, easier, more
> seductive..."

yeah, and i'm mostly here because a bunch of folks who are otherwise
pretty advanced users struggled to "make the ramyeon". (not helped, i
think by the whole "wtf? is anyone still using this stuff in 2023?"
problem that means the web is probably less helpful than usual.)

> > (though i'd still probably have had the ctrl-] ? help
> > output be one per line, but perhaps you wouldn't have been as offended
> > by the verbosity if the user had _asked_ for help?)
>
> If ctrl-] does NOT exit into an enclosing shell script with a case statement
> prompting you to send a file or similar, then microcom has a modal interface
> like vi. Unless microcom has hotkeys for each action (which is what function
> keys were for but alas desktops started intercepting those so they're useless
> now, and most of the ALT-X and so on that don't produce an ascii code have been
> similarly hijacked by greedy GUIs), then you need an obvious visual indicator
> that you're now in "command mode".
>
> As long as you've got that visual indicator anyway, it might as well convey
> "what can you DO in this command mode?" Having four lines of output do what one
> line of output could do seemed weird to me.

yeah, just our usual disagreement of whether "shorter" and "more
readable" are the same or different :-)

> >> > (and, if nothing else, fix the previously
> >> > inconsistent spacing between options
> >>
> >> Yup, my bad. :)
> >>
> >> > and make it clear what new options
> >> > should look like).
> >>
> >> I'd rather not apply the patch as is because of the above commit comment
> >> grumbles, but don't object to code change itself. Should I switch the -64 to &31
> >> while I'm there?
> >
> > (i still think we should do that properly as you described --- let's
> > have the macro, but have it do the _right_ thing and `^ 0x20` first.)
>
> If replacing three character -64 with 6 characters of macro wrapper seems like
> the right thing to you... ok?

i think, like the above arguments, a lot of it comes down to my usual
"don't make me think". (or, in the less snappy but more folksy version
favored by a guy i used to work with, "the one thing i can guarantee
is that i won't be smarter tomorrow than i am today, so i should try
to make things easy on future me".)

> I'm not really thrilled about including the header
> which defines a lot of land mines like CEOF and CMIN and that seem easy to hit
> accidentally, but if glibc's already leaking this into the namespace then we've
> already taken the hit and might as well explicitly #include the header. Modulo
> not breaking mac and bsd so maybe it needs to be in portability.h, I haven't
> looked...

that header's in macOS too, but like i keep saying --- i'm actually
suggesting (based on your "what about case?") that we have a _better_
one where it doesn't matter because the macro does the right thing
regardless.

> >> Rob
> >>
> >> P.S. Oh goddess, I just "grep -ir modem busybox/" and yes busybox has an "rx"
> >> command. But it doesn't have a corresponding sx command, which makes every
> >> little sense? Who sends it files?
> >>
> >> https://unix.stackexchange.com/questions/186706/what-is-the-complementary-command-to-rx-for-xmodem-transfer
> >
> > indeed. that's why i'd add rx _and_ sx, and have microcom know how to run sx.
> >
> > (it would help with being able to at least do some round-trip testing too!)
>
> Ctrl-]
> sx file.txt /dev/ttyS0
> cursor up twice, enter.
>
> That's the traditional way.

i suspect https://github.com/tio/tio#32-key-commands is probably
making new traditions right now. well, that and PuTTY! tbh, i only
ever see people on youtube using gui stuff; there's even a gtkterm!

(interestingly, i see tio also only has xmodem and ymodem. i think
zmodem is just too complex for what it does to have any value any
more.)

> But seriously... xmodem?

well, when we get that time machine, we can add "include a file size,
because cp/m isn't going to last long" to our to-do list... :-)

> Rob