[Toybox] ls implementation

[Rob Landley]

On 02/14/2012 02:15 PM, Andre Renaud wrote:
> Hi Rob,
> 
>>> I've added a couple of bits to it, so that it supports ls -lFa1. The
>>> main thing missing is support for files in the command line (currently
>>> it only supports directories).
>>
>> And -R support.
> 
> Attached is a patch that adds support for having files on the command
> line (as well as directories), and -R support. There are two drawbacks
> to it however:
> 
> The files on the command line always get treated as separate items, so
> they always appear one per line, even if -1 hasn't been specified.
> 
> To support -R I've done the trivial recursive implementation, but
> because I need a string for the next directory name, it has pretty
> massive stack usage (PATH_MAX * depth).
> 
> Both of these issues would be resolved by maintaining some kind of
> growing list of entries to be written to, which could be re-purposed as
> they're printed, however the structure I implemented initially doesn't
> extend to that trivially.
> 
> For now, I think this is relatively usable & simple, if not particularly
> optimal.

Looks like you've got a pretty good handle on the issues, I merged this
and am happy to stay out of your way on ls for the moment.

I point you at toys/cp.c and the lib/dirtree.c stuff it uses. I now
think my old dirtree stuff is designed a bit wrong, but you can see what
I was getting at.

My rule of thumb for "toybuf" usage is that any command is free to use
toybuf at any time for anything, meaning if _library_ files need to use
it the command should pass it in (so the library is never overwriting
its contents behind the command's back: otherwise calling library
functions could magically screw up toybuf as an unexpected side effect,
which would be bad).

I initially had dirtree using toybuf internally anyway because it's
recursive, and thus passing toybuf to itself over and over would eat 4
bytes of stack space each time.  (And passing a buffer _and_ a length
would eat 8, or 16 on 64-bit platforms, which starts to add up.

The reason I cared so much about stack size is back then I was trying to
humor nommu systems, which can't dynamically grow the stack _and_ need
contiguous memory for it in a potentially fragmented shared heap,
meaning it tends to be small (64k or even smaller) both so it doesn't
limit the number of simultaneous processes you can have and so the
system can easily find a congiguous chunk that big (otherwise it can't
launch any new processes).

I went back and cleaned dirtree's use of toybuf[] out again, because
it's a library function and they shouldn't refer directly to toybuf
(although there are still some traces of toybuf in the current
lib/dirtree.c because I wasn't that thorough cleaning it out), and
instead made it so the initial path has to be in a writeable PATH_MAX
size buffer (I.E. copy it into toybuf yourself before calling; slightly
magic calling conventions but...).

The reason I want this as common code is that cp, mv, find, tar, ls -R,
and so on all have to do recursive operations on directories, and it's
more or less the same stuff.  The dirtree thing could either create a
binary tree in memory containing the directory contents (useful for
gene2fs which needs to write out the metadata before it writes out the
file data: it can pad with zeroes if files go away but can't go back and
change its' mind about _which_ files it's writing), or it can do a
callback on each file with the path of the file to handle, a bit like
loopfiles().

I'm still not happy with the PATH_MAX size limitation since Linux itself
is now unlimited (255 bytes max per filename, but directories can nest
arbitrarily deep).  And this uses readdir() while yours uses scandir(),
and I agree scandir() is designed to implement ls (we get some of the
sort code in libc for free...)

Anyway, there needs to be some design work done here, and I haven't
necessarily got time to do it just now.  I'm basically asking your
opinion on what you think this code _should_ look like?

> Regards,
> Andre

Thanks,

Rob

 1329601764.0