[Toybox] vi 'b' command broken

Fri Oct 6 01:49:34 PDT 2023

On Thu, Oct 5, 2023 at 11:15 PM enh via Toybox <toybox at lists.landley.net> wrote:
>
> On Wed, Oct 4, 2023 at 11:51 PM Rob Landley <rob at landley.net> wrote:
> >
> > On 10/4/23 21:51, Oliver Webb wrote:
> > > ------- Original Message -------
> > > On Wednesday, October 4th, 2023 at 18:59, Rob Landley <rob at landley.net> wrote:
> > >> On 10/4/23 13:51, enh via Toybox wrote:
> > >>
> > >> > (since it looks like there are folks actively working on vi atm...)
> > >> >
> > >> > looks like 'b' goes to the end of the previous word, rather than
> > >> > the beginning of the current word?
> > >>
> > >>
> > >> There's no 'b' but there is a "b" (which is weird because all the vi_mov_param
> > >> are chars so why is that a string) which dispatches to vi_movb() which is
> > >> multiplying count0 by count1. What ARE count0 and count1?
> > >
> > >>From my looking over of the vi.c, count0 and count1 specify arguments to commands
> > > ("15G" gives 15 as count0 in vi_go()). As for the multiplying, all my debug printf's
> > > show count1 being set to 1 so I have no idea why it's multiplying.
> >
> > Ah. command0 is the number before the command, command1 is the number after the
> > command. If you ":5d" it deletes the 5th line. If you ":d5" it deletes 5 lines
> > starting from the current one. Presumably if you ":5d5"... yup, it deletes 5
> > lines starting from line 5.
>
> ah, that's why i was confused by two counts --- i didn't know there
> was a postfix count too. (i'd wondered whether they were just bad
> names for the two numbers in something like :1,5s/// instead!)
>
> > Before I wrote my own "sed" I didn't really know how to use sed. Before I wrote
> > my own "find" I know like 1/3 of the things find could do.
>
> judging by some of the sed in the toybox build, you know more sed than
> macOS' sed does :-)
>
> > And the reasong
> > writing my own bash takes so long is I'm learning ALL SORTS of weird corner
> > cases of bash behavior.
> >
> > Here's a fun one I learned today:
> >
> > $ bash -c 'set -e;x() { echo one;false;echo two; }; x || echo three; x'
> > one
> > two
> > one
> >
> > Being on the left side of the || operator disables set -e (exit on error) in a
> > way that permeates INTO shell functions. I need to add a test for that. (And
> > then implement it.)
> >
> > The main reasons I haven't cracked open bc and awk and vi and so on yet is I
> > need to learn to USE them at a level I've never previously needed to. I taught
> > an "intro to unix" course in a previous life which spent two classes on vi and
> > taught the students something like 15 commands which would be on the test. That
> > was "goddess I'm old" many years ago, I remember that "set mark" exists but not
> > what it did. But even then I only taught the students a smattering of what vi
> > can do, and all these years later I remember maybe half of what I taught. (I use
> > a tiny subset of the available commands. Mostly I just type fast.)
> >
> > Luckily my model here isn't vim, it's probably busybox vi. In fact ubuntu's
> > intentionally sabotaged vi (because mark shuttleworth prefers emacs) that
> > doesn't let you cursor around in insert mode until you "sudo ln -f vimrc
> > /etc/vim/vimrc.tiny" (which is part of my install checklist when I upgrade
> > debian versions; I never apt-get dist-update, I back up my home dir and do a
> > fresh install so unreproducible debris doesn't accumulate where my magic system
> > works and nobody else does because I have a 12 year config tweak under
> > /var/lib/pam disabling some stupid default behavior I've long forgotten about,
> > and also so I know the last time I didn't just backup but RESTORED from backup.)
> >
> > And of course the main downside of wanting to implement the subset of vi that
> > busybox implements is their LOVELY documentation:
> >
> > $ ./busybox --help vi
> > BusyBox v1.37.0.git (2023-10-02 07:51:31 CDT) multi-call binary.
> >
> > Usage: vi [-c CMD] [-R] [-H] [FILE]...
> >
> > Edit FILE
> >
> >         -c CMD  Initial command to run ($EXINIT and ~/.exrc also available)
> >         -R      Read-only
> >         -H      List available features
> >
> > And if you run it and :help it says not implemented. So yay. It does, for some
> > reason, have :features which says:
> >
> > These features are available:
> >         Pattern searches with / and ?
> >         Last command repeat with .
> >         Line marking with 'x
> >         Named buffers with "x
> >         Some colon mode commands with :
> >         Settable options with ":set"
> >         Signal catching- ^C
> >         Job suspend and resume with ^Z
> >         Adapt to window re-sizes
> >
> > For whatever help that is.
>
> "some colon mode commands" is especially good. i mean, why even bother
> saying anything if you're going to be that vague?
>
> > Anyway, opening the vi can of worms? If Elliott says it's suddenly on his
> > critical path, I can pivot.
>
> no, i don't care, and i don't think many others do either. i only
> mentioned this because someone else mentioned it, and i was surprised
> that it wasn't one of those things that's easier to just send a patch
> than send a bug report.
>
> > Until then, ignoring the shell for the moment, and
> > the expr/$((math))/dc/bc hairball not sharing any code, and od/xxd/hd/hexedit
> > not sharing any code, and of course "grep pending Android.bp":
> >
> >     # Edit the relevant `srcs` below, depending on where the toy should be
> >     "toys/pending/diff.c",
> >     "toys/pending/expr.c",
> >     "toys/pending/getopt.c",
> >     "toys/pending/tr.c",
> >     "toys/pending/brctl.c",
> >     "toys/pending/getfattr.c",
> >     "toys/pending/lsof.c",
> >     "toys/pending/modprobe.c",
> >     "toys/pending/more.c",
> >     "toys/pending/stty.c",
> >     "toys/pending/traceroute.c",
> >     "toys/pending/vi.c",
> >
> > But there's _structural_ stuff I'm halfway through and should finish. I'm
> > partway through automating a LFS build for mkroot that I can feed toybox
> > commands into to test with real world package builds. I want to switch the zlib
> > library inclusion stuff to use weak symbols instead of this #ifdef stuff.
> >
> > I've already checked in the basic lib/passwd.c rewrite but still need to migrate
> > the md5/sha1/sha256/sha3 plumbing into lib where it's internally reusable
>
> speaking of which, https://www.phoronix.com/news/Mold-2.2-Linker
> mentioned blake-3. and funnily enough, if i'm understanding
> https://reviews.llvm.org/D121531 correctly, i think llvm's lld
> silently uses blake-3 if you ask for md5 (which i think is the
> default; certainly it's what Android is using). nothing actionable
> here, i only bring it up because of the recent "is anyone using this?"
> conversation. ("yes, quite heavily, but not in a way that would
> warrant adding a toy".)
>
> > (because glibc is being stupid about crypt) and test/fix/promote chsh, groupadd,
> > groupdel, sulogin, useradd, and userdel, plus re-review su, passwd, login, and
> > mkpasswd. And of course the md5sum.c/sha3sum.c commands that need their plumbing
> > transplanted...
> >
> > Oh, and I think the libc api got yanked out from under the "who" and "w"
> > commands again, got a todo item about that somewhere. And I have a bunch of
> > patches like 0001-ulimit-actually-use-the-units-we-claim.patch that turned into
> > a thing trying to deal with them which I need to cycle back to with some clear
> > desk space...
> >
> > > I took a crack at this once I saw Elliot's email thinking this would be a trivial fix without rewriting
> > > the "b" command. But after looking over vi_movb(), I can barley tell why half the code is there
> > > (There is a for loop only explained by a comment "find first".
> > > When I comment the entire for loop out the I can use the "b" command as normal...)
> >
> > My problem isn't figuring out what the code is doing, my problem is figuring out
> > what the proper behavior of "vi" is to the various inputs. Alas, "man vim" only
> > describes command line options and then references "vimtutor" (which doesn't
> > exist) and says to run the :help command which pulls up a "this is not
> > installed!" page on debian because an ascii text file the size of the bash man
> > page (man bash | gzip -9 | wc is 98k) would be a crazy thing to actually install
> > with your 1.2 megabyte binary.

On vim there is :help word-motions which opens up motions.txt file.
Which has some rather cryptic documentation about words, WORDS
and inclusive and exclusive moves....

What it does not explain is why and how motions behave differently
when it is combined with yank, delete, change or no operation....

Also just by trying motions inside this read protected file on vim.
why does "b" still execute the movement when giving it file modifying
command in read protected file but "w" does not....

some versions of motion.txt and man page for vi can be found online
https://vimhelp.org/motion.txt.html
https://man7.org/linux/man-pages/man1/vi.1p.html

In my experience busybox vi movements are bit wonky too
compared to nvi and vim.

Then after fixing vi_movb() I would test it with:
./vi tests/files/utf8/test2.txt

To ensure utf8 support didint break.

> >
> > I'm tempted to try to put together some tests for the existing vi (which busybox
> > vi would also presumably pass, and maybe even debian's) using txpect() and "vi
> > >/dev/null" or similar where I diff before/after files to see that the requested
> > changes got made to the file when I fed it commands. Probably a vixpect wrapper
> > the same way tests/sh.test has shxpect that brackets it in a useful
> > load-the-file do the stuff save-the-file-and-exit wrapper. (With each command
> > running under a 10 second timeout, of course...)
> >
> > But I need to rewrite mcm-buildall.sh to build the glibc+musl toolchains without
> > using musl-cross-make (which Rich hasn't pushed a commit to in a year and a
> > half), and all that stuff with the hexagon toolchain
> > (https://www.openwall.com/lists/musl/2023/09/27/2) and I should link to that
> > "trusting trust" post I made from the about.html page and move the section on
> > the old toybox logo to the FAQ page now that there is one, and help.html doesn't
> > use the nav bar wrapper like the other pages do, and the other day I tried to
> > build "hello world" with linux's nolibc because I was thinking maybe some subset
> > of toybox might someday be made workable with that but:
> >
> >   $ cat hello.c
> >   #include <nolibc.h>
> >   int main(int argc, char *argv[]) { return write(2, "Hello world\n", 12); }
> >   $ gcc -nostdinc -nostdlib -fPIC -I tools/include/nolibc hello.c \
> >     -I walrus/include -I include/linux
> >   /usr/bin/ld: /tmp/ccC05cSk.o: relocation R_X86_64_32S against symbol `environ'
> >    can not be used when making a PIE object; recompile with -fPIC
> >   /usr/bin/ld: final link failed: nonrepresentable section on output
> >   collect2: error: ld returned 1 exit status
> >
> > Which led to https://lwn.net/Articles/920158/ which was too long to read at the
> > time and I never DID get a cortex-m target working with qemu so I don't have
> > pull out a physical board to test nommu (in theory coldfire would work too but
> > that's also nontrivial) and I can't build new versions of QEMU without python
> > 3.8 which means upgrading from devuan botulisum to devuan diptheria which means
> > closing all the open windows (or waiting for a crash or battery exhaustion but
> > uptime -s says my last reboot was march 25 because Linux) and I _still_ need to
> > properly fix cp -s for the three cases and when did I last post my kernel
> > patches to linux-kernel (it's like calling my senators in Texas: futile, but I
> > like to think it annoys them) and I should start cutting a release (two commands
> > promoted, something to show for it at least) and that idea about using the $PATH
> > to indicate toybox recursion (the traditional empty entry means search the
> > current directory but it COULD mean search within toybox, which would easily
> > specify whether to run toybox commands before the filesystem ones or after, but
> > I remember there was something hinky when I tried it the other night, there's
> > probably an open window that would trigger my memory if I could figure which one
> > and if all else fails try it again and reproduce the problem...) and I still
> > need to fix pgrep "name with space" and rm -r still isn't doing infinite
> > recursion yet (because filehandle exhaustion, it needs to do the .. climbing
> > trick with either the drill down from top retry or error_exit() which raised the
> > "there's no global state for dirtree descent" issue I think I blogged about?)
> > and I'm going to lose access to github when they insist on attaching my phone
> > number to the account for tracking purposes (no) and did I ever fix that
> > overlayfs issue
> > http://lists.landley.net/pipermail/toybox-landley.net/2022-September/029195.html
> > which reminds me I should try to get overlayfs working in mkroot, and I wanted
> > to get cgi and virtual domains working in httpd (and some hardening like length
> > limits and timeouts on the line reads), and if I'm going to have stuff that
> > needs an inetd I should _have_ an inetd (that works on nommu) instead of using
> > netcat server mode to test it, and I've got to cycle back to diff.c and finish
> > that, and mount --make-rprivate and friends, and go through all the tests
> > without the executable bit set (toybox find tests -maxdepth 1 \! -executable) to
> > see which ones should be promoted into "make tests"...
> >
> > Ahem. So yeah, resisting poking too hard at vi just now. It might fall on me.
> >
> > Rob
> _______________________________________________
> Toybox mailing list
> Toybox at lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net

-Jarno