[Toybox] What should I do with my boxes generic editor code?

[David Seikel]

Well, the evening I had said I was gonna reply got eaten up by four
unexpected visitors in a row.  Time to do so now.

On Tue, 26 May 2015 15:31:57 -0500 Rob Landley <rob at landley.net> wrote:

> On Tue, May 26, 2015 at 12:08 AM, David Seikel <onefang at gmail.com>
> wrote:
> > This has been most frustrating.
> >
> > tl,dr - Getting boxes into toybox has entirely failed for four
> > years, which has blocked the progress of both.  Let's do something
> > about that.
> 
> I suspect the best thing to do is just release it as an independent
> package. I've been consistently overwhelmed with stuff, merging as
> fast as I can.

Yep, looking like that's gonna be the way to do so.  Perhaps if I can
get some decent progress doing things that way, we might be able to
merge some parts between the two projects.

Most of the rest of your reply, that I have read now, is just showing
that I'll need to clean it up a lot to make it Rob friendly.  Now that
I have your initial comments, that will be easier to do.  B-)

> Hand #defining FLAG_ macros even though the infrastructure's done
> this for us for at least a year.

To be fair, this was written four years ago, before that infrastructure
existed.  Needs an update.

> "The edit line is basically a movable readline"... not really, no.
> For command history I've been pondering how to handle "I scrolled up
> and got a long multiline command history thing and then scrolled past
> that to get a shorter one, so even if I scroll back down it's now
> several lines _up_ from the bottom of the screen because the
> transient long line moved everything". Lines that wrap can vary in
> size, they're no more _one_ vertical line than horizontal characters
> are one character with utf-8. Speaking of which, searching this file
> for "mbr" or "wcwidth" didn't find any hits? I added the basic
> fontmetrics dance to hexedit's filename display in commit
> 5ea14bd1c246 a couple weeks ago, not because it's important for a hex
> editor but because I'm using a simple command to work out the
> infrastructure I want to genericize. Your version has a lot of very
> complicated behavior but doesn't address fundamental issues I'm
> interested in getting right, ones which aren't necessarily easier to
> retrofit on _top_ of all that complexity.

The code only does "long lines scroll left and right" for now.  Dealing
with "shells use multiple lines to edit long lines" and UTF-8 is extra
complexity in something that's already overly complex as a starting
position just to get it in.  Yes, I intended to add that stuff later,
but trying to get this cut down yet still functional first.

> At line 2400 there's two screens full of what looks like a man page
> cut and pasted into the middle of the code? (Why not just reference
> the man page?)

Ah, just notes to myself about which of those terminal settings I
wanted to use.  Not just copy pasta, it has a sprinkling of oregano as
well.

> Ok, backwards search for "main(" and... it's got a multiplexer for
> "mode", with emacs, joe, less, mcedit, more, nano, and vi. But you
> can't submit one at a time, because this code cannot be broken up
> into smaller pieces. Right.

I wrote it as a generic editor, that is table driven.  So it includes
tables for the stuff toybox is interested in.  Vi is in fact the only
one with extra code besides the tables.  I could have submitted it as
just the vi toy, that would only have left out the extra tables.  It
might have been my next step, if handlekeys had got in.

> Ok, let's look at vi, which is the editor required by posix. Case
> insensitive search for "vi " from the top of the file... help text,
> help text, comment, comment... ok, the eleventh hit is still in yet
> another comment (right after yet another TODO) but it's right in
> front of "simpleEditCommands" which sounds like fun, and which seems
> to have built-in help text? "Back space last character." Ok, let's
> see what displays this help text. Forward searching from
> simpleEditCommands it's in simpleLess (is there a non-simple less?)
> and in simpleMore and in simpleMcedit... ok, "simple" here seems to
> be an assertion rather than actually informative.

During my research I had split editing functions into groups, the first
one is "simple", which is anything that's needed to be able to just
barely be able to edit things.  Load a file, cursor around, insert,
delete, overwrite, save the result.  This group has been implemented.

The next group of commands adds the usual basic tools that most editors
have, and is called "basic".  That group would be what I start on next
once "simple" has been bedded down.  The other two groups are
"advanced" and "code", advanced editing tools, and stuff that's
specific to source code editing.  These last two may or may not be
suitable for toybox, though might end up cherry picking some of that.

> What are the _other_ files here? README.md: "Please don't actually
> include this in toybox." BOXES.txt: a series of disconnected
> notes-to-self apparently not meant to be read by anybody other than
> its author.

Yep, BOXES.txt is just my own collected notes, not something I wrote to
be read by others.  Someone asked for my "design docs", which is what
BOXES.txt is.  So I just dumped that into the repo for them, then
explained to them that it's just my notes, ability to decipher it all
was up to them.

> handlekeys.c is 445 lines of code. my lib/interstingtimes.c and
> toys/*/hexedit.c combined are 133+286 = 419 lines, I.E. smaller than
> that. I'm not saying mine's better (you handle sigwinch, I implement
> a hex editor), I'm just saying that in terms of breaking off
> manageable chunks of code from boxes to add to toybox I've never had
> a clue where to _start_.

handlekeys.c is the bit to start from, dumbsh.c is an example of using
handlekeys.

> > Me completing it by fleshing out the missing editing
> > functions wont make it any more bite sized, that has to wait until
> > it's in toybox.  Once it is in, fleshing it out CAN be done in bite
> > sized bits.
> 
> Once I install emacs you can write small lisp snippets on top of it?

I have no intention of adding Lisp to boxes.  That goes beyond my four
groups of editing functions, way beyond.  If I could be convinced to
add any sort of scripting language, it would be Lua.

> > So, what are my choices?
> >
> > 0) Try to convince Rob to just dump it into pending.  Then we can
> > all work on it to make it great.
> 
> Has this worked for the rest of pending? (Also, I haven't got a
> pending for lib or tests, which is a bit of a structural problem I
> haven't thought of a good way to fix yet.)

Yeah, bits of this should be in lib, so having a pending for lib and
tests would be good.

> > 1) Keep waiting another four years in the forlorn hope that Rob
> > actually does something with it.
> 
> It's not a question of the amount of time. I do not know how to
> proceed with the existing codebase, and trying to guilt me into
> accepting code in its present form isn't a lot of fun from this end
> either.

Wasn't trying to guilt you into this.  Though I am a bit frustrated by
the lack of progress, and that might leak through.  I did try to not
let it leak through, even managed to keep all but one swear word out of
it.

> > 3) Write it as a drop in module for toybox.
> 
> I thought that was the original plan?

Nope, just a temporary condition until it got in.

> > Another thing to bring up is that I had discussed on this mailing
> > list before my desire to implement Midnight Commander (mc) as part
> > of toybox. Something that Rob agreed to in principle at least.
> 
> Yeah, it's one of the things I was thinking might be a logical next
> step after hexedit. The problem is I haven't used xtgold since dos,
> and never really used mc (a clone of xtree, which I believe was
> originally for Sun or SGI or some such).

As I mentioned, I use mc constantly.  It's my IDE of choice.  Pretty
much first thing I do on any box I'm looking after is to make sure mc
is installed on it.  I've used xtgold a long long time ago, and maybe
even xtree, but mc is what I would be cloning.

> It's also not a priority since it's not required by any standards or
> build enviornments, and many other remaining todo items are. Adding
> hexedit let me work out display infrastructure, which basically took
> a couple of weekends. There isn't similar collateral upside with mc,
> it would just be a second user of that infrastructure and I could do
> command history editing or "less" instead, both of which have aspects
> of those wordwrap variable line height issues.

In a similar way, editors could use a file requester for loading a
file, save as, insert file, save block to file, etc.  Once I have
written a file requester, put two of them side by side, hook up
function keys to the mv and cp commands in toybox, and you got the
beginnings of a mc clone.  Then hook up more function keys to less and
$EDITOR (which could be one of the ones in toybox, and would default to
mcedit, coz that's the mc editor).  That covers 80% of what I do with
mc, I'd probably switch over at that stage.  Then use it to develop the
other 20%.  B-)

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20150529/797cbbea/attachment-0002.pgp>