[Toybox] [PATCH] microcom: simple menu, new "paste file" functionality.

[Rob Landley]

Blah, forgot to click "send" and let the window get buried...

On 10/28/23 12:18, enh wrote:
> On Sat, Oct 28, 2023 at 12:29 AM Rob Landley <rob at landley.net> wrote:
>>
>> On 10/25/23 19:48, enh via Toybox wrote:
>> > Rather than take up yet another key for the new functionality, I've moved
>> > everything to a menu, similar to the one in telnet. It's not amazing,
>> > but it's the least code I could get away with, and it's good enough for
>> > now, and it leaves us in a better place if we need to add more stuff
>> > (such as xmodem, say).
>> >
>> > But this is enough to let me stop using minicom and get back to toybox
>> > microcom for today!
>>
>> What I always did was "toybox uuencode target_filename < local_file | xclip
>> -selection clipboard" and then run uudecode at the far end and right
>> click->paste it into the relevant xterm window. I hadn't noticed a size limit...

Sorry, applied this yesterday but didn't push. (My maybe-covid has turned into
conventional cold with sore throat and stuffed up nose and everything, and I'm a
little out of it right now...)

> yeah, so recently i had one group ask for xmodem and another group who
> just wanted to use base64. the latter was the thing i could do in half
> an hour :-)

xmodem? <obiwan>Not that's a name I haven't heard in a long time.</obiwan>

Sheesh, If you're going to do any of that use zmodem. We were mad at xmodem back
on the C64: latency spike waiting to ack each 128 byte packet, rounding up the
file size to 128 bytes, single byte simple sum error detection, no standard way
to abort a partial transfer without leaving the recipient hanging...

(And if you're going to have any nontrivial transfer protocol, why not do what
all the term programs did and call a child executable you hand the serial
connection to so it can do the transfer, then exit back into the terminal
program. That way you're not hardwiring many protocols into your term program.)

The only times I've had a corrupt file across serial in the past 20 years were
because of an insufficiently buffered UART dropping bytes, which was an OS
latency spike problem not a serial hardware problem. And I say that with a
UART->ttl cable right next to me because I need to plug it into the orange pi 3b
I've been gradually setting up as a server. Unfortunately the shortest one I
have is 6 feet and I'd like maybe a 3 inch one because the kitchen counter space
next to the router is a finite resource...

> (others had no idea how to transfer a file over serial at all. no-one
> came up with your solution _that i know of_ but maybe this is what the
> people who complained that "doing the paste myself is flaky"?)

My solution evolved from "cut and paste from the terminal" to "cut and paste
from mousepad" to "actually learn to use the command line clipboard tool" over a
multi-year period of fiddling with boards that have serial interfaces. (Or qemu
system emulation with /dev/console connected to stdin/stdout as an emulated
serial device, which is how mkroot sets everything up and aboriginal linux
before it.)

In theory I could sha1sum the file afterwards and check the first couple and
last couple bytes not being nonsense, but in practice I haven't had a corrupted
file in forever. It's a bit like not running on rad-hardened hardware: yes we
COULD SECDED all the memory and checksum/resend the memory bus transactions, but
these days it's hard to even buy parity memory because the voltage of the signal
and capacitance of the wire are way above what a cosmic ray of alpha particle is
gonna do. (And if you somehow get one of those giga-giga-whatever jumbo cosmic
rays deciding to pop inside your electronics, which should never happen, with
modern microtrace sizes you now have a cut wire.)

>> There was discussion of a CTRL() macro in context of vi, but I don't see where
>> it was applied to toys.h or lib/lib.h? Ah, it's in glibc's sys/ttydefaults.h
>> which is included from termios.h... but not in the NDK build, or in the musl
>> build. It _only_ leaks in the glibc build.
> 
> ah, oops. i did check that everyone (including macOS) _has_
> <sys/ttydefaults.h>, but i didn't check that it was getting
> [transitively] included everywhere.
> 
>> The macro's actually a bit non-obvious to me because CTRL(A) and CTRL(a) seem
>> like they should produce different results? Ah, and they want CTRL('A').
> 
> yeah, if we did our own macro i'd suggest `#define whatever(ch) (((ch)
> ^ 0x20) - '@')` to avoid that. there doesn't seem to be a strong
> consensus about whether it's more natural to say A or a, so may as
> well make both work.

Or just &31 which is what I probably should have done. Assuming CTRL-SHIFT-X
isn't a thing we care about. (I tried ctrl-shift-h at the command line and it
did a backspace, so apparently not...)

>> What was wrong with 'A'-64 again?
> 
> that's a variant i _haven't_ seen yet. 'A'-'@' seems more common, but
> it's not as readable to n00bs.

I went with the first 3 character fix off the top of my head. &31 is also 3 chars.

>> You want "p) paste file" instead of "f) send file"? (Paste-but-not-clipboard has
>> more mnemonic weight than File?)
> 
> having used this for a week, i intend to send you another patch that
> changes 'e' for exit to 'q' for quit because that's been annoying,

*shrug* Sure.

> but
> i don't have a strong opinion on 'p'. i went with that because it was
> the "more direct" of minicom's _two_ ways to send a file as "ascii"
> (terminology i disliked for obvious reasons).

I haven't used minicom in ages. Busybox grew a microcom back in 2007, before
that I was using netcat -f.

> it sounds like the cool kids use https://github.com/tio/tio these
> days, and https://github.com/tio/tio#32-key-commands shows that it
> doesn't have an equivalent of this functionality anyway.
> 
>> The read(1) loop is similar to the loop in lib/passwd.c:read_password() and I
>> should probably factor it out.
> 
> yeah, and similar to vi and hexedit. this was the minimum i needed
> right now, though, and i thought you'd have strong opinions on what
> exactly to factor out anyway, given that the shell is going to need
> something readline-like.
> 
>> You only accept DELETE not BACKSPACE to shorten
>> the input line?
> 
> yeah, and i didn't do arrow keys or home/end or anything else;

In xfce my backspace key produces ctrl-h, my delete key produces ascii 127. Sun
microsystems had it the other way around in 1992. I vaguely recall there's a way
to change the mapping in the tcsetattr)() nonsense, but I've seen backspace be
"remove character to the left of the cursor" and delete be "remove character to
the right of the cursor" for a longish time.

I don't remember what KDE or Gnome did, it's been years since I used either
outside of VMs and guest logins on other people's machines...

> nor did
> i deal with "what happens when you have more input than fits on one
> line?" ... in the week i've used this, _tab_ (for file completion) was
> the thing i would have actually liked, but not enough to warrant
> implementing it, and especially not before the inevitable [but hard to
> predict timing of] factoring out!

Somewhere I said I have a big todo item for shell line editing with all the
bells and whistles. I should bump that up...

>> You have a progress indicator instead of letting the other end do that. (How big
>> are the files you're sending?)
> 
> well, i'm using this for `base64 -d -i > file` so the other side isn't
> going to help me here. (and a quick glance at the xmodem docs suggests
> that one of the limitations of that protocol is that it doesn't
> include file size, so even there i think the _sender_ is going to have
> to do the work.)
> 
> given that the only use i see for this is "your board can't even do
> adb yet", the likelihood [for me] is that i'm having to send static
> binaries. after seeing how long a 20MiB [well, ~10MiB, but base64
> bloats it up] binary took (two meetings + cooking and eating a meal),
> i did make an effort to trim my binary down to just what i actually
> needed to iterate on,

I provide static toybox binaries linked against musl for a _reason_. :)

What do you have your serial rate set at? Anything modern should be able to do a
megabit no problem even with a ten foot cable, the old boards that maxed out at
115,200 were running at 16mhz. Modern uarts have all that bit edge detection
clock resyncing stuff which is why there's still a 1 in the 8n1 to force a
periodic transition even when you're sending all zeroes or ones. (Even within
chips, clock domain crossings looked like tiny half-assed UARTs to me.)

> but even so it was a couple of MiB each time.
> "long enough that i favored one of minicom's two alternatives for this
> functionality based on the fact that it showed me progress, but not
> long enough that i bothered to implement a countdown/ETA".

I've been meaning to factor out the new count -l logic into lib somewhere. I
just wanted a bit more thought on the hardwired logic ala 4/sec update
granularity and 8/16 (I forget) seconds trailing history in the rate. (Rate of
the whole thing is a trivial size/ticks calculation at the end if we want that...)

A second user for "progress indicator" is interesting. And I suppose wget would
be a third. Both those know how long the total should be (which COULD be a
command line argument for count, or I could have -l do an fdlength() on stdin to
see if we're reading from something it can detect length for...)

Almost every command has more work I COULD do on it, I just try to reach a good
stopping point and prioritize...

>> Rob

Rob