[Toybox] thoughts on `tail -F`?

[Rob Landley]

On 6/22/21 7:33 PM, enh via Toybox wrote:
> "how hard can `tail -F` be?" i thought ... but unless i'm using inotify wrong,
> the answer turns out to be "quite a lot harder than i expected because i need
> watch the parent directory for a different set of events, and then filter those".

Never used it before,

> strace tells me that coreutils actually does use inotify, whereas busybox just
> polls (by sleeping). would you be happy (at least for the time being) with a
> crappy polling `tail -F`?

Sure.

It's not even that crappy, how much response time granularity do you want?
(Dunno the expected use case, if it's humans 1/second is plenty, "watch"
defaults to every 2 seconds, any "less" implementation dumb enough to listen to
posix is batching same-line output frustratingly. In theory inotify on the
directory can wake up _more_ often if it's an active enough directory. Whatever
you do, a file that's rapidly created and deleted again can be missed by another
process, And if your tail -c output size is finite then a file that starts with
a large enough atomic write will make tail skip data...

I remember Linus once said on linux-kernel that Linux won't split a write()
going to a "local" filesystem, by which I assume he meant "block backed" in the
block/pipe/ram/synth categories from http://landley.net/toybox/doc/mount.txt .
(Presumably ram backed is in there too. Pipe backed filesystems are only atomic
to PIPE_BUF, which includes network filesystems, and synthetic filesystems can
do anything the driver cares to implement including gratuitously short writes.)

But block backed filesystems won't unless they fill up, so a 1 gigabyte write()
atomically updating the file length is a real possibility. Note that Mac at one
time did _not_ handle atomicity even to the level posix requires (the write()
can be shorter than requested by what DOES get accepted, everybody sees the same
thing at the same time):

https://stackoverflow.com/questions/10650861/atomicity-of-write2-to-a-local-filesystem

Anyway: yay polling implemenation of -F.

> i can guarantee i can find the time to do that...
> rewriting the inotify stuff to support deletion/re-creation keeps failing to fit
> into any available time gap (and then i have the "where was i?" problem that i
> know you're all too familiar with!).

Oh yeah. I refer to it as swap thrashing. Scheduler overhead and faulting data
back in eats your time slice.

Rob