[Toybox] [PATCH] date: some fixes.

[enh]

On Mon, Feb 11, 2019 at 9:35 PM Rob Landley <rob at landley.net> wrote:
>
>
>
> On 2/11/19 2:15 PM, enh wrote:
> > On Fri, Feb 8, 2019 at 10:18 AM Rob Landley <rob at landley.net> wrote:
> >>
> >>
> >>
> >> On 2/8/19 11:54 AM, enh wrote:
> >>>
> >>>
> >>> On Fri, Feb 8, 2019, 09:25 Rob Landley <rob at landley.net <mailto:rob at landley.net>
> >>> wrote:
> >>>
> >>>     On 2/7/19 9:08 PM, enh via Toybox wrote:
> >>>     > Add support for more input formats, primarily the ISO formats used by
> >>>     > the AOSP build.
> >>>
> >>>     Ok.
> >>>
> >>>     > Also, our interpretation of @UNIXTIME was wrong: surprisingly, it should
> >>>     > respect $TZ.
> >>>
> >>>     That's _insane_.
> >>>
> >>>     Does "date +%s" also adjust for $TZ? If so, it's NOT UNIXTIME. If it does, "date
> >>>     @$(date +%s)" moves the clock by multiple hours...
> >>>
> >>>
> >>> This change certainly produces results consistent with the GNU date, but I
> >>> suspect it's not "right right".
> >>
> >> <facepalm>sigh</facepalm>
> >>
> >>> I didn't get to a point where I had working
> >>> support for input starting `TZ="blah" `
> >>
> >> Starting as in setting the environment variable on the command line...?
> >
> > no, that's what i mean by this not being "right right" (i.e. it's a
> > local maximum, not the global maximum) ... my patch gets us to where
> > we're no worse than busybox, but GNU date actually supports something
> > like:
> >
> > TZ=Europe/London date -d 'TZ="America/Los_Angeles" @1549649764'
>
> That's even more insane.
>
> Sigh. This is the part of the patch I object to:
>
> -    gmtime_r(&tt, tm);
> +    // Somewhat surprisingly, even Unix input times respect $TZ.
> +    localtime_r(&tt, tm);
>
> The rest of it I'd have applied already. I'm aware GNU is doing something
> stupid, and I understand you have AOSP build issues that depend on the GNU
> stupid. I just... I'm facepalming here in real life. There was a loud dramatic
> sigh. Out loud.

it actually does make sense. i think this implementation doesn't make
much sense, but the true "two timezones" GNU implementation does...

> > where there are two separate time zones in play. (and this is used in
> > the AOSP build. there are a couple of [failing] examples in the
> > tests.)
>
> Tell you what, lemme rip those 3 lines out of your patch and apply the rest now
> to at least get the delta down. I expect I'm just going to have to lump it and
> do the stupid thing GNU does because existing scripts depend on The Stupid, but...
>
> How does unix time not mean UTC? How do they implement +s output not using
> timezones and @input applying a timezone offset? (Answer: The FSF is bad at
> software and cares nothing for consistency. But it still hurts.)

i'm pretty sure (despite having failed repeatedly for weeks to find
the time to really get to the bottom of this and fix it) that in this
case the *input* is taken as UTC but the GNU implementation *also* has
an output conversion that toybox and busybox are missing. so this is a
quick hack that gives equivalent results for the cases we do support
until such time as we actually support both timezones.

in the meantime, when trying to understand this i think it makes more
sense to stop thinking about the special case of Unix times. imagine a
regular ISO date instead. basically what GNU lets you do _in one step_
is say "if it's $IN_TIME in timezone $TZ1, what time is it in $TZ2?".
with `TZ=$TZ2 date -d 'TZ="$TZ1" $IN_TIME'`.

like i said, there are examples at the end of the tests file. (and
note that the first two are subtly different to cover both daylight
and non-daylight savings.)

> Huh, 4 of your new tests are failing with TEST_HOST=1 ?

man, you really *never* read my comments in tests, do you? :-P

> FAIL: date -d 06021234
> FAIL: date -d 060212341982

these are actually your tests, but moved around and commented to
explain why they fail on the host :-)

...i did want to talk to you about them, but i thought (correctly, as
it turned out) that the two timezones issue was going to be confusing
enough to be getting on with. but i did explain these failures in new
test comments:

# TODO: these are rejected by coreutils and interpreted differently by busybox.
# busybox thinks this should use the current year, not 1900.
testing "-d 06021234" "TZ=$tz date -d 06021234 2>&1" "Sun Jun 2
12:34:00 UTC 1900\n" "" ""
# busybox thinks this is the year 603 (ISO time 0602-12-34 19:82 with
out of range fields normalized).
testing "-d 060212341982" "TZ=$tz date -d 060212341982 2>&1" "Sun Jun
2 12:34:00 UTC 1982\n" "" ""

fwiw, i think we should reject this format [that i've never seen
before], just like coreutils does. but if you added this because
busybox added it because it is actually useful somewhere, then we
should probably be consistent with busybox (except busybox appears to
be insane, even if they're the only other tool i know that supports
this at all)! remove?

> FAIL: date -d 1110143115.30
> FAIL: date -d 111014312015.30

these are a bit different. again, coreutils rejects these, but i
totally understand why toybox supports them (because they're the usual
POSIX date *setting* format), and toybox's interpretation of these is
logically consistent with POSIX. i just didn't know how to say "skip
these two on the host because ...".

# POSIX format with 2- and 4-digit years.
# TODO: coreutils rejects POSIX format supplied to -d.
testing "-d 1110143115.30" "TZ=$tz date -d 1110143115.30 2>&1" "Sun
Nov 10 14:31:30 UTC 1915\n" "" ""
testing "-d 111014312015.30" "TZ=$tz date -d 111014312015.30 2>&1"
"Sun Nov 10 14:31:30 UTC 2015\n" "" ""

> Hmmm, VERBOSE=fail says...
>
> FAIL: date -d 06021234
> echo -ne '' | TZ=Europe/London date -d 06021234 2>&1
> --- expected    2019-02-11 23:23:26.734146212 -0600
> +++ actual      2019-02-11 23:23:26.738146212 -0600
> @@ -1 +1 @@
> -Sun Jun  2 12:34:00 UTC 1900
> +date: invalid date ‘06021234’
>
> >>> so I backed out what I'd done and sent
> >>> you this as a "better than yesterday" stop gap.
>
> I have applied a subset of it. Probably apply the rest tomorrow, but I need time
> to mourn first.
>
> This is _really_ stupid. I mean this is from the man page:
>
> EXAMPLES
>        Convert seconds since the epoch (1970-01-01 UTC) to a date
>
>               $ date --date='@2147483647'
>
> They SAY the epoch is in UTC and this is seconds since that! Right there in the
> man page! They're inconsistent with their own man page!

i don't think they are. i think you just have to read the whole man
page and realize that although they treat this as UTC on the way in,
they'll convert it to $TZ on the way out. and my hack says "in the
absence of two conversions, take $TZ into account in the one
conversion we do do". so, yeah, gross, but a local maximum and "better
than busybox".

> >>> But I suspect that when we get
> >>> to the point where we handle separate input and output timezones, this
> >>> presumably goes back to UTC (unless it's preceded by TZ=) and the *output*
> >>> conversion produces this effect instead?
>
> *blink* *blink* What?
>
> $ date ; date --date=@$(date +%s)
> Mon Feb 11 23:32:21 CST 2019
> Mon Feb 11 23:32:21 CST 2019
>
> I don't understand what's going on here. That's what
>
> > but like i say, to be _better_ than busybox we need to take both time
> > zones into account, and that probably needs us to stop using `struct
> > tm` internally, and do two conversions.
>
> Sigh. Send me a patch to make the AOSP build work and I'll apply it. I don't
> have to _like_ it.
>
> But I don't think I currently understand the expected behavior...
>
> $ date ; date --date=@$(TZ=UTC date +%s)
> Mon Feb 11 23:33:58 CST 2019
> Mon Feb 11 23:33:58 CST 2019
> $ date ; TZ=UTC date --date=@$(date +%s)
> Mon Feb 11 23:34:15 CST 2019
> Tue Feb 12 05:34:15 UTC 2019
>
> No, I _really_ don't understand what's going on here.
>
> >> Rob
>
> Rob