[Toybox] [PATCH] microcom: move from e[x]it to [q]uit.

[Rob Landley]

On 11/3/23 09:44, enh wrote:
> On Fri, Nov 3, 2023 at 7:22 AM Rob Landley <rob at landley.net> wrote:
>>
>> On 11/2/23 16:56, enh via Toybox wrote:
>> > This frees up 'x' for xmodem transfers later.
>>
>> Q to quit makes sense, adding xmodem... do you have an xmodem protocol extension
>> in mind that doesn't pad the last packet up to 128 bytes?
> 
> that is annoying, yes, but since my only purpose is transferring
> executables, i don't think it matters? (and even in the case where it
> does, toybox already has a truncate(1).)

A transfer protocol known to modify files is not ideal.

>> What xmodem command
>> are you going to run at the other end to consume the data and send the replies,
> 
> i was going to write one. the xmodem spec sounds like it's only going
> to be tens of lines.

There's a spec? It's like 3 status bytes and 128 byte packets... (I was on
bulletin boards when I was 12. There's a nonzero chance I implemented it back on
the commodore 64, and even then it was inferior to punter.)

>> and how will the terminal program know to run it?
> 
> i think for unix implementations of xmodem it's more common for the
> user to do that part? the xmodem spec does talk about having sx send
> "rx\r" and rx ignoring crap like that at the start, but that seems
> gross enough to wait until someone actually _needs_ that.

Busybox never bothered to implement the sx command, so I can't check there, but
I would expect them to do that because "I am at a command prompt, send file"
would just work. :)

Except for the part where xmodem doesn't know what the filename is. I'm guessing
it would write a.out by default?

>> I still lean towards running a child command here. Heck, _vi_ has that (for some
>> reason), I know because I keep hitting it accidentally.
> 
> yeah, like i said earlier when we were talking about the progress
> indicator --- "call out to sx(1)" is the obvious way to implement
> xmodem, so replacing my custom "cat-alike with progress indicator"
> with _actual_ cat(1) wouldn't be unreasonable (even if i'd be a little
> sad to lose my progress indicator).

I vaguely intend to move the new granular count.c progress indicator into lib,
possibly removing -l from count and just having it always do that.

A bar graph mode isn't hard to do, and there's a bargraph() in ps.c that could
also get genericized and moved to lib...

>> Whether we then add a
>> zmodem command to toybox is a separate question...
> 
> *z*modem is way more complicated (and still kind of a mess).

How about ymodem? Ymodem was an attempt to fix the problems with xmodem: 1k
packet size since we were no longer on 300 baud modems, block 0 includes the
filename and size so it doesn't pad the last block, and 16-bit CRC instead of 8
bit checksum. If you're going to do a file transfer protocol from the 1980s BBS
world, ymodem is... much less bad.

Zmodem was the BBS-era's streaming serial protocol, meaning it doesn't insert
latency spikes between each packet waiting for the far end to acknowledge, but
instead constantly sends data until it runs out of data. (It also doesn't modify
the files it's sending.)

I also note that I've been meaning to write an rsync.c for toybox, and under the
covers rsync is another member of this "send files across a serial connection"
family. It's usually tunneled over ssh but it's just "bidirectional pipe". And
like zmodem, it sends data to the far end until it runs out of data to send,
asynchronously reading return results and appending updates to its stream as it
processes them. So no latency spikes waiting for the other side to reply to what
it said, except at the very end waiting for the far side to say it's done.

I.E. the next logical step past zmodem is rsync. The next logical step BEFORE
zmodem is ymodem.

By the way, there are UI questions we're just kinda glossing over here. If you
want "send these files to the far end" to be integrated... that may be a call to
rsync. We haven't got a midnight commander variant that lets you select files
and get a file list. The traditional unix way of doing this would call the
various tools in a shell script, with the output of one being the input to (or
command line arguments of) another.

I believe busybox microcom's expected usage is you can exit microcom without
closing the connection, type "sz file.txt" at the command line, and then cursor
up twice in your command history and hit enter to get microcom back. I switched
the exit character from ctrl-X to ctrl-] because somebody might want to use
emacs and ctrl-] is what telnet used back on the sun workstations in 1992.

The sharp edge is knowing sz should write to /dev/ttyACM3, which is why people
made wrapper shell scripts, and the potential justification for "type child
process command line and I'll run it with stdin/stdout going to the serial
connection". There was also an environment variable solution at some point, I think?

But that's why they didn't historically build xmodem or similar into the
terminal program, because if microcom can exit and let you do stuff and then
resume, you're not limited to what they bothered to integrate into the program.
You can start up a slip/ppp session or something if you need to. But if you
start integrating the functionality _into_ the program, you've never got enough
options and it goes fractal fast.

>> > Also, if we're going to have a single line of help, switch to the more
>> > common format for such prompts
>>
>> More common where? All the hits for grep '"[^"]*[[].[]][^"]*"' toys/*/*.c
>> lib/*.c main.c are false positives finding array access between two strings. I
>> don't think I've seen toybox use this anywhere else?
> 
> oh, yeah, since my unzip(1) is c++ and uses Android's libziparchive i
> only upstreamed the unzip _tests_ to toybox, didn't i?
> 
> in terms of _toybox_ precedent, well, that's why i had the original
> multi-line format :-) (toybox telnet uses multiple lines because
> _busybox_ telnet does.) i can't think of any other interactive
> prompting in toybox.

I'm not against having it, I just want to know what the basis is. :)

> but (as i just spoilered), unzip is about the only "unix" tool (in the
> weaker sense of "something most unix users have come across at some
> point" anyway)

Because it started life in DOS. Abort/Retry/Ignore. From there it became the
default windows archiver (because Windows a coat of paint on top of DOS until
the W2K problem), and was adopted by linux via the open source info-zip
implementation I first used under OS/2 in ~1992.

> that has an interactive prompt (when it needs to
> overwrite something),

$ touch pluvial
$ rm -i pluvial
rm: remove regular empty file 'pluvial'?
$ touch pluvial
$ toybox rm -i pluvial
rm pluvial (y/N):

Not saying it's good, just saying it _is_. That's showing your options in
parentheses, because the gnu/dammit one not showing your options seems less user
friendly. (Toybox has terse output because I don't assume the reader speaks
english, "name of command you just ran can plausibly be interpreted as a verb,
name of file, keys you can hit in reponse". I _also_ try not to assume they have
experience with the posix command line, because there's a first time for everybody.)

This isn't conceptually the same as "[a] thingy [b] more [c] whatsis" but for a
sufficiently LONG list, I personally find "a) option b) addendum c) walrus"
makes it clear that the letter goes with which description, not "If you build it
he will come. To have him build it for you, press [1]."

> and it uses this style. and it's certainly more
> readable than the "busybox telnet squished onto one line" style, and
> the fact that the checked-in spacing between options wasn't even
> correct seemed like a strong indicator to me that "this is not the way
> to do it" :-)

A typo indicates which style is preferable?

I haven't cleaned up telnet because I use dropbear and netcat. Last time I
touched it other than treewide API changes looks like commit 755e040916d9 in
2014 and the only prompt that changed was:

-  if(TT.port <= 0 || TT.port > 65535)
-    error_exit("PORT can be non-zero upto 65535.");
+  if(TT.port <= 0 || TT.port > 65535) error_exit("bad PORT (1-65535)");


>> Then again, grep '"[^"]* .)[^"]*"' toys/*/*.c lib/*.c main.c  isn't finding any
>> other non-false-positive hits. If you have a strong opinion about this, I can
>> switch, but I just want to be clear that either way this seems to be introducing
>> a new rule into the project, and you want that rule to be [a] command rather
>> than a) command.
> 
> well, i'm actually suggesting "[c]ommand" rather than "c) command".

Which with e[x]it raises the the issue of breaks between multi-word descriptions.

> but, yes, i think if you prefer "c) command" then we should actually
> switch to "toybox --help" style where it's _usually_ one per line (and
> there's a whole _tab_ for clear and consistent spacing) ...
> until/unless you end up with lots of lots of options, at which point
> things get a bit ascii-art.

I wanted to collate them together on a line because it's an inefficient use of
space otherwise, and scrolling stuff off the top of the screen seems impolite.

We need _a_ prompt, and we've cleared the next line, so we might as well make
use of that space we just reserved for ourselves.

We could instead do the "snapshot cursor position and jump to a reserved line"
except we don't reserve a line, and jumping to the top might not be noticed and
jumping to the bottom means scrolling the screen which we wouldn't want to do
multiple types if they typo (since you don't have an ESC key but instead treat
any unknown key as ESC with another line of message.)

At which point I start going "I started writing not-curses code in hexedit.c and
then a second user in less.c and I was going to flesh it out properly in vi.c
and have watch.c share it... and if microcom ever needed anything more
complicated that was also a potential user... except I didn't get to write vi. I
was too slow. And the people who did write the one we are now committed to use
didn't know about my infrastructure plans and where else I wanted to reuse
common code. Which is part of the reason I haven't started the shell command
line editing stuff yet, because that was ALSO tied into this thing that's
missing a limb now.

Oh well...

> (_that_ "toybox --help" style was actually my first thought for toybox
> telnet too, if i'm honest, but either choice seemed like "innovation"
> and i fell on the side of "same ui as busybox" being "less
> innovative". which then naturally led to me doing the same for
> microcom.)

I haven't looked much at pending/telnet.c. I haven't _used_ telnet it forever, I
use ssh or netcat. Does it also have user interface parts like this?>

I used telnet fairly heavily in college on sun workstations, but only a few
times since. Early on I hit a case where opening a network socket to a telnet
server, running "tar" at the far end, and then feeding it a tarball wound up
with corrupted files because it was treating some of the data as escape
sequences, and rather than reverse engineer why I installed ssh...

> my other other thought (which i also didn't pursue because i liked the
> telnet precedent of the "Escape character ..." line) was outputting
> something like
> 
> toybox microcom
> Press ctrl-] q to quit. Press ctrl-] ? for help.
> 
> on startup.

Which I would not expect users to remember 30 second after it's scrolled off,
and which also adds in-band signaling to something that's otherwise transparent
so you can do the exit-and-wrap thing. (That's why busybox's microcom also works
like that.)

I tend to have unix sensibilities, where "I'm gonna use this command in a shell
script" is the default assumption. A big chunk of your userbase is coming over
from windows where everything is gui (no two commands ever connect together,
that would be a license violation) and they expect something more like:

  https://clip.cafe/who-framed-roger-rabbit-1988/laughing-pull-the-lever/t/1/

  https://pinterest.com/pin/324822191849854340

One community dials phone numbers and the other has a contact list. One
navigates to a destination and the other has elevator buttons for every known
destination. One has ingredients, the other has microwaveable dinners.

Reality is usually somewhere in between. I have leanings, which I am willing to
be talked out of, but I worry about an ever-growing pile of special cases, vs
I've had the same hammer for 30 years and learned lots of different ways to use it.

I readily admit your first experience being "here's some flour and water" vs
"microwaveable ramen" is a big difference. As Yoda said, "Quicker, easier, more
seductive..."

> (though i'd still probably have had the ctrl-] ? help
> output be one per line, but perhaps you wouldn't have been as offended
> by the verbosity if the user had _asked_ for help?)

If ctrl-] does NOT exit into an enclosing shell script with a case statement
prompting you to send a file or similar, then microcom has a modal interface
like vi. Unless microcom has hotkeys for each action (which is what function
keys were for but alas desktops started intercepting those so they're useless
now, and most of the ALT-X and so on that don't produce an ascii code have been
similarly hijacked by greedy GUIs), then you need an obvious visual indicator
that you're now in "command mode".

As long as you've got that visual indicator anyway, it might as well convey
"what can you DO in this command mode?" Having four lines of output do what one
line of output could do seemed weird to me.

>> > (and, if nothing else, fix the previously
>> > inconsistent spacing between options
>>
>> Yup, my bad. :)
>>
>> > and make it clear what new options
>> > should look like).
>>
>> I'd rather not apply the patch as is because of the above commit comment
>> grumbles, but don't object to code change itself. Should I switch the -64 to &31
>> while I'm there?
> 
> (i still think we should do that properly as you described --- let's
> have the macro, but have it do the _right_ thing and `^ 0x20` first.)

If replacing three character -64 with 6 characters of macro wrapper seems like
the right thing to you... ok? I'm not really thrilled about including the header
which defines a lot of land mines like CEOF and CMIN and that seem easy to hit
accidentally, but if glibc's already leaking this into the namespace then we've
already taken the hit and might as well explicitly #include the header. Modulo
not breaking mac and bsd so maybe it needs to be in portability.h, I haven't
looked...

>> Rob
>>
>> P.S. Oh goddess, I just "grep -ir modem busybox/" and yes busybox has an "rx"
>> command. But it doesn't have a corresponding sx command, which makes every
>> little sense? Who sends it files?
>>
>> https://unix.stackexchange.com/questions/186706/what-is-the-complementary-command-to-rx-for-xmodem-transfer
> 
> indeed. that's why i'd add rx _and_ sx, and have microcom know how to run sx.
> 
> (it would help with being able to at least do some round-trip testing too!)

Ctrl-]
sx file.txt /dev/ttyS0
cursor up twice, enter.

That's the traditional way. But seriously... xmodem?

Rob