[Toybox] vi 'b' command broken

[Rob Landley]

On 10/4/23 21:51, Oliver Webb wrote:
> ------- Original Message -------
> On Wednesday, October 4th, 2023 at 18:59, Rob Landley <rob at landley.net> wrote:
>> On 10/4/23 13:51, enh via Toybox wrote:
>> 
>> > (since it looks like there are folks actively working on vi atm...)
>> > 
>> > looks like 'b' goes to the end of the previous word, rather than
>> > the beginning of the current word?
>> 
>> 
>> There's no 'b' but there is a "b" (which is weird because all the vi_mov_param
>> are chars so why is that a string) which dispatches to vi_movb() which is
>> multiplying count0 by count1. What ARE count0 and count1? 
> 
>>From my looking over of the vi.c, count0 and count1 specify arguments to commands
> ("15G" gives 15 as count0 in vi_go()). As for the multiplying, all my debug printf's
> show count1 being set to 1 so I have no idea why it's multiplying.

Ah. command0 is the number before the command, command1 is the number after the
command. If you ":5d" it deletes the 5th line. If you ":d5" it deletes 5 lines
starting from the current one. Presumably if you ":5d5"... yup, it deletes 5
lines starting from line 5.

Before I wrote my own "sed" I didn't really know how to use sed. Before I wrote
my own "find" I know like 1/3 of the things find could do. And the reasong
writing my own bash takes so long is I'm learning ALL SORTS of weird corner
cases of bash behavior.

Here's a fun one I learned today:

$ bash -c 'set -e;x() { echo one;false;echo two; }; x || echo three; x'
one
two
one

Being on the left side of the || operator disables set -e (exit on error) in a
way that permeates INTO shell functions. I need to add a test for that. (And
then implement it.)

The main reasons I haven't cracked open bc and awk and vi and so on yet is I
need to learn to USE them at a level I've never previously needed to. I taught
an "intro to unix" course in a previous life which spent two classes on vi and
taught the students something like 15 commands which would be on the test. That
was "goddess I'm old" many years ago, I remember that "set mark" exists but not
what it did. But even then I only taught the students a smattering of what vi
can do, and all these years later I remember maybe half of what I taught. (I use
a tiny subset of the available commands. Mostly I just type fast.)

Luckily my model here isn't vim, it's probably busybox vi. In fact ubuntu's
intentionally sabotaged vi (because mark shuttleworth prefers emacs) that
doesn't let you cursor around in insert mode until you "sudo ln -f vimrc
/etc/vim/vimrc.tiny" (which is part of my install checklist when I upgrade
debian versions; I never apt-get dist-update, I back up my home dir and do a
fresh install so unreproducible debris doesn't accumulate where my magic system
works and nobody else does because I have a 12 year config tweak under
/var/lib/pam disabling some stupid default behavior I've long forgotten about,
and also so I know the last time I didn't just backup but RESTORED from backup.)

And of course the main downside of wanting to implement the subset of vi that
busybox implements is their LOVELY documentation:

$ ./busybox --help vi
BusyBox v1.37.0.git (2023-10-02 07:51:31 CDT) multi-call binary.

Usage: vi [-c CMD] [-R] [-H] [FILE]...

Edit FILE

	-c CMD	Initial command to run ($EXINIT and ~/.exrc also available)
	-R	Read-only
	-H	List available features

And if you run it and :help it says not implemented. So yay. It does, for some
reason, have :features which says:

These features are available:
	Pattern searches with / and ?
	Last command repeat with .
	Line marking with 'x
	Named buffers with "x
	Some colon mode commands with :
	Settable options with ":set"
	Signal catching- ^C
	Job suspend and resume with ^Z
	Adapt to window re-sizes

For whatever help that is.

Anyway, opening the vi can of worms? If Elliott says it's suddenly on his
critical path, I can pivot. Until then, ignoring the shell for the moment, and
the expr/$((math))/dc/bc hairball not sharing any code, and od/xxd/hd/hexedit
not sharing any code, and of course "grep pending Android.bp":

    # Edit the relevant `srcs` below, depending on where the toy should be
    "toys/pending/diff.c",
    "toys/pending/expr.c",
    "toys/pending/getopt.c",
    "toys/pending/tr.c",
    "toys/pending/brctl.c",
    "toys/pending/getfattr.c",
    "toys/pending/lsof.c",
    "toys/pending/modprobe.c",
    "toys/pending/more.c",
    "toys/pending/stty.c",
    "toys/pending/traceroute.c",
    "toys/pending/vi.c",

But there's _structural_ stuff I'm halfway through and should finish. I'm
partway through automating a LFS build for mkroot that I can feed toybox
commands into to test with real world package builds. I want to switch the zlib
library inclusion stuff to use weak symbols instead of this #ifdef stuff.

I've already checked in the basic lib/passwd.c rewrite but still need to migrate
the md5/sha1/sha256/sha3 plumbing into lib where it's internally reusable
(because glibc is being stupid about crypt) and test/fix/promote chsh, groupadd,
groupdel, sulogin, useradd, and userdel, plus re-review su, passwd, login, and
mkpasswd. And of course the md5sum.c/sha3sum.c commands that need their plumbing
transplanted...

Oh, and I think the libc api got yanked out from under the "who" and "w"
commands again, got a todo item about that somewhere. And I have a bunch of
patches like 0001-ulimit-actually-use-the-units-we-claim.patch that turned into
a thing trying to deal with them which I need to cycle back to with some clear
desk space...

> I took a crack at this once I saw Elliot's email thinking this would be a trivial fix without rewriting
> the "b" command. But after looking over vi_movb(), I can barley tell why half the code is there 
> (There is a for loop only explained by a comment "find first". 
> When I comment the entire for loop out the I can use the "b" command as normal...)

My problem isn't figuring out what the code is doing, my problem is figuring out
what the proper behavior of "vi" is to the various inputs. Alas, "man vim" only
describes command line options and then references "vimtutor" (which doesn't
exist) and says to run the :help command which pulls up a "this is not
installed!" page on debian because an ascii text file the size of the bash man
page (man bash | gzip -9 | wc is 98k) would be a crazy thing to actually install
with your 1.2 megabyte binary.

I'm tempted to try to put together some tests for the existing vi (which busybox
vi would also presumably pass, and maybe even debian's) using txpect() and "vi
>/dev/null" or similar where I diff before/after files to see that the requested
changes got made to the file when I fed it commands. Probably a vixpect wrapper
the same way tests/sh.test has shxpect that brackets it in a useful
load-the-file do the stuff save-the-file-and-exit wrapper. (With each command
running under a 10 second timeout, of course...)

But I need to rewrite mcm-buildall.sh to build the glibc+musl toolchains without
using musl-cross-make (which Rich hasn't pushed a commit to in a year and a
half), and all that stuff with the hexagon toolchain
(https://www.openwall.com/lists/musl/2023/09/27/2) and I should link to that
"trusting trust" post I made from the about.html page and move the section on
the old toybox logo to the FAQ page now that there is one, and help.html doesn't
use the nav bar wrapper like the other pages do, and the other day I tried to
build "hello world" with linux's nolibc because I was thinking maybe some subset
of toybox might someday be made workable with that but:

  $ cat hello.c
  #include <nolibc.h>
  int main(int argc, char *argv[]) { return write(2, "Hello world\n", 12); }
  $ gcc -nostdinc -nostdlib -fPIC -I tools/include/nolibc hello.c \
    -I walrus/include -I include/linux
  /usr/bin/ld: /tmp/ccC05cSk.o: relocation R_X86_64_32S against symbol `environ'
   can not be used when making a PIE object; recompile with -fPIC
  /usr/bin/ld: final link failed: nonrepresentable section on output
  collect2: error: ld returned 1 exit status

Which led to https://lwn.net/Articles/920158/ which was too long to read at the
time and I never DID get a cortex-m target working with qemu so I don't have
pull out a physical board to test nommu (in theory coldfire would work too but
that's also nontrivial) and I can't build new versions of QEMU without python
3.8 which means upgrading from devuan botulisum to devuan diptheria which means
closing all the open windows (or waiting for a crash or battery exhaustion but
uptime -s says my last reboot was march 25 because Linux) and I _still_ need to
properly fix cp -s for the three cases and when did I last post my kernel
patches to linux-kernel (it's like calling my senators in Texas: futile, but I
like to think it annoys them) and I should start cutting a release (two commands
promoted, something to show for it at least) and that idea about using the $PATH
to indicate toybox recursion (the traditional empty entry means search the
current directory but it COULD mean search within toybox, which would easily
specify whether to run toybox commands before the filesystem ones or after, but
I remember there was something hinky when I tried it the other night, there's
probably an open window that would trigger my memory if I could figure which one
and if all else fails try it again and reproduce the problem...) and I still
need to fix pgrep "name with space" and rm -r still isn't doing infinite
recursion yet (because filehandle exhaustion, it needs to do the .. climbing
trick with either the drill down from top retry or error_exit() which raised the
"there's no global state for dirtree descent" issue I think I blogged about?)
and I'm going to lose access to github when they insist on attaching my phone
number to the account for tracking purposes (no) and did I ever fix that
overlayfs issue
http://lists.landley.net/pipermail/toybox-landley.net/2022-September/029195.html
which reminds me I should try to get overlayfs working in mkroot, and I wanted
to get cgi and virtual domains working in httpd (and some hardening like length
limits and timeouts on the line reads), and if I'm going to have stuff that
needs an inetd I should _have_ an inetd (that works on nommu) instead of using
netcat server mode to test it, and I've got to cycle back to diff.c and finish
that, and mount --make-rprivate and friends, and go through all the tests
without the executable bit set (toybox find tests -maxdepth 1 \! -executable) to
see which ones should be promoted into "make tests"...

Ahem. So yeah, resisting poking too hard at vi just now. It might fall on me.

Rob