[Toybox] thoughts on `tail -F`?

[enh]

On Wed, Jun 23, 2021 at 3:35 AM Rob Landley <rob at landley.net> wrote:

> On 6/22/21 7:33 PM, enh via Toybox wrote:
> > "how hard can `tail -F` be?" i thought ... but unless i'm using inotify
> wrong,
> > the answer turns out to be "quite a lot harder than i expected because i
> need
> > watch the parent directory for a different set of events, and then
> filter those".
>
> Never used it before,
>

me neither. apparently testy folks ... no, that's not the right word :-)
... *QA* folks like to rotate logs, and doing that breaks `tail -f` like
the rest of us use.


> > strace tells me that coreutils actually does use inotify, whereas
> busybox just
> > polls (by sleeping). would you be happy (at least for the time being)
> with a
> > crappy polling `tail -F`?
>
> Sure.
>
> It's not even that crappy, how much response time granularity do you want?
> (Dunno the expected use case, if it's humans 1/second is plenty, "watch"
> defaults to every 2 seconds, any "less" implementation dumb enough to
> listen to
> posix is batching same-line output frustratingly. In theory inotify on the
> directory can wake up _more_ often if it's an active enough directory.
> Whatever
> you do, a file that's rapidly created and deleted again can be missed by
> another
> process, And if your tail -c output size is finite then a file that starts
> with
> a large enough atomic write will make tail skip data...
>
> I remember Linus once said on linux-kernel that Linux won't split a write()
> going to a "local" filesystem, by which I assume he meant "block backed"
> in the
> block/pipe/ram/synth categories from
> http://landley.net/toybox/doc/mount.txt .
> (Presumably ram backed is in there too. Pipe backed filesystems are only
> atomic
> to PIPE_BUF, which includes network filesystems, and synthetic filesystems
> can
> do anything the driver cares to implement including gratuitously short
> writes.)
>
> But block backed filesystems won't unless they fill up, so a 1 gigabyte
> write()
> atomically updating the file length is a real possibility. Note that Mac
> at one
> time did _not_ handle atomicity even to the level posix requires (the
> write()
> can be shorter than requested by what DOES get accepted, everybody sees
> the same
> thing at the same time):
>
>
> https://stackoverflow.com/questions/10650861/atomicity-of-write2-to-a-local-filesystem
>
> Anyway: yay polling implemenation of -F.
>

cool! meetings ate today, but i have high hopes for tomorrow (and an
implementation from someone else that's probably fine)...


> > i can guarantee i can find the time to do that...
> > rewriting the inotify stuff to support deletion/re-creation keeps
> failing to fit
> > into any available time gap (and then i have the "where was i?" problem
> that i
> > know you're all too familiar with!).
>
> Oh yeah. I refer to it as swap thrashing. Scheduler overhead and faulting
> data
> back in eats your time slice.
>

(speaking of which, although it sounds like a terrible self-help book, i
actually quite enjoyed "Algorithms to Live By: The Computer Science of
Human Decisions". i don't think i've read a pop-sci book on CS before!
anyway, yeah, they tried to introduce the term "thrashing" [in our sense]
to the general population...)


> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20210623/807f4a55/attachment-0001.htm>