[Toybox] [PATCH] Support the %N coreutils/busybox extension to date.

[Rob Landley]

On 04/16/2017 03:51 PM, enh wrote:
> yeah, definitely a mess.
> 
> note that i only did this because (a) existing tests were using it and
> (b) busybox has it too. i've been assuming (perhaps incorrectly) that
> busybox also tried to avoid needlessly implementing GNU extensions
> unless they were actually useful to enough people.

As far as I can tell the way people do this is indeed date %N. I've
tried to do this before, but adding extra escapes to both input and
output via an overlay mechanism turns out to be a hard problem to do right.

(Really I want a generic mechanism to wrap arbitrary printf/scanf style
things, but the problem space isn't necessarily set up that way. Can I
chop strptime parsing into multiple strptime calls that _only_ modify
certain fields each time while leaving the others alone? I was reading
the posix page and man page stuff about that at one point, something
said it only modified specific fields other than the ones you
specified... but alas, I got distracted halfway through...)

Checking my notes, I was attempting to add %s support to the input path.
Except that %s pretty much _replaces_ strptime() because none of the
other fields you set make sense in combination with that. (But %s could
be in the middle of a context string describing the rest of the input;
that I didn't make up, it's in the ubuntu one.) So %s is probably
fiddlier than %N. Hmmm...

In general I wince at merging half a solution to a larger problem,
because it increases the surface area of the problem. (People start
using the partial thing, then I break them when I try to fix the rest of
it...) That said, if I'm not doing the full fix now and the partial fix
is there and we need it... Sigh.

>     Speaking of which, this patch adds %N to output but not to input, so
>     date can still only _set_ nanoseconds using the @123456789.123456789
>     syntax.
> 
> isn't setting via arbitrary format a toybox extension? 

Walter Harms added it to busybox in 2006 (back when I maintained it):

  http://lists.busybox.net/pipermail/busybox/2006-February/018203.html
  https://git.busybox.net/busybox/commit/?id=c5789a6e234c

I genericized his original idea a bit because "I want to convert data
from one format to another" is awkward to solve otherwise. "This email's
datestamp would be what in unix time"... I used to write little C
programs to do that.

I used to have a button that said "Only XT users know that January 1,
1980 was a tuesday". Toybox says:

  $ ./date -D "%m %d %Y" -d "1 1 1980" +%A
  Tuesday

If you can say %N in + but not in -D I'm doing it wrong. It's a tiny
thing, but it bothers me.

>     And you're not checking that "%%N is a literal %N". (Sigh. If
>     we're going there, may I draw your attention to next_printf() in
>     lib/lib.c? Example usage in seq and stat.)
> 
> okay, i'll switch it over (assuming we actually want %N).

Eh, we probably want it, I just winced at how expensive it was to
implement and it's the wrong place to do it (belongs in libc) and if
we're going there I want to do the full fix (get and set, and handling
%% and %4N) so it isn't inconsistent...

And it needs one of them Focused Review things that have been in short
supply recently. (The easter weekend's helped me to dig out a bit. If I
got a three day weekend every week, keeping up would be so much easier. :)

>     It's kind of sad "ls" can't show nanoseconds. Or "stat". I understand
>     why (nanoseconds were added when makefile dependencies started breaking
>     as computers got faster and multiple build things happened in the same
>     second; nothing else actually _needed_ them in the filesystem) but it's
>     still half-assed.
> 
> ~$ ls -ld --time-style=full-iso /

--time-style is a can of worms (I made a design decision _not_ to
support more than one output format, now there's a straightforward
extension of the existing output format we want to add, but if I use the
generic named selection mechanism for it the pressure to support all the
rest of 'em will never stop. I lose the ability to draw a clear boundary
line.)

But --full-time is a synonym, which I can add _and_ make -ll do it too.
(Repeat -l and it shows nanoseconds. I like having short options for
things, but the letters are all taken here. Still, logically it's a
"long long"... :)

My concern here is maximizing the surface area of "I can write a script
that runs under toybox and under ubuntu as well." Such as the current
test suite stuff we're trying to do. I can write one using --full-time
it works in both contexts.

The problem with using ls for the touch test code is "ls -l" shows
ownership information also, varying between systems. You can't output
_just_ the info you want without that. We can do the awk/cut trick to
filter it and only look at specific fields (todo: can we have spaces or
tabs in usernames? You can in the realname field...).

tl;dr there's still a call for "date %N".

> i'm happy to rewrite the tests instead... i've never used %N personally,
> and would just naturally use stat for this kind of thing.

How do you get nanoseconds out of stat? (I'm happy to add a toybox
extension for it, but relying on that breaks TEST_HOST or "make
test_sed" on a non-toybox host.)

> as you say, the asymmetry is ugly, and the divergence from libc is ugly,
> so if busybox added this "just because" maybe we shouldn't have it at all?

It was useful, and still is. (Especially the ability to specify
different input and output formats so you can convert between them.) And
it's already implemented and working. :)

Rob