[Toybox] tar --null

[Rob Landley]

On 7/25/22 14:58, enh wrote:
> On Tue, Jul 19, 2022 at 6:34 AM Rob Landley <rob at landley.net
> <mailto:rob at landley.net>> wrote:
>     >     Good suggestion, but I'm never sure what actually _is_ easy. I shelved
>     this
>     >     after doing about half of mp3 identification, which turns out to be a
>     >     surprisingly large rathole due to funky container formats. (And I
>     don't trust
>     >     anything microsoft's ever touched not to be turing-complete to solve...)
>     >
>     > heh, i know exactly what you mean because (a) i have this problem all the time
>     > at work, where people don't finish their "starter project" for years and (b)
>     > your specific jpeg size example was one _i_ punted when i originally submitted
>     > jpeg support because it turned out to be non-obvious.
> 
>     The problem with "leaving easy stuff for other people to do" is they don't do
>     it. I submitted a series of updated patches to make the kernel's
>     CONFIG_DEVTMPFS_MOUNT work for initramfs and not just the fallback root= mount,
>     and nobody else ever picked that up and put it in.
> 
>     https://lkml.org/lkml/2017/9/13/651 <https://lkml.org/lkml/2017/9/13/651>
> 
>     That seems quite easy, no? Here it is again 3 years later...
> 
>     https://lkml.iu.edu/hypermail/linux/kernel/2005.1/09399.html
> 
>     There's a type of salesmanship in getting Huck Finn's friends to paint his fence
>     which is a completely different skillset from doing the work yourself. It's a
>     quite useful skill I do not have.
> 
> yeah, but at the same time we do get people asking "is there something i can do
> to help?", and having a pile of fences to paint eases that.

I am terrible at maintaining such piles. :)

(If it looks simple and I haven't already done it, generally there's some subtle
sharp edge. Doesn't mean it was a SIGNIFICANT problem, just what made me put it
down where a drift of papers blew over it...)

>     > i still think this is the "least worst" option though, and that's actually one
>     > reason why i suggested a separate bug for each: it lets people thrash about a
>     > bit until they find one that _is_ easy (for them).
> 
>     People have been trying to get me to do more with bug trackers for a larger
>     number of years than I like to think about.
> 
>     There's a whole lot of years of unmedicated ADHD tangled up in there. I use them
>     when somebody ELSE manages them and regularly reviews what's been sitting there
>     composting. 
> 
> yeah, sorry for not having stayed on top of that after the last time you
> mentioned it was an issue. i've been through just now and closed out the stuff
> where we think it's fixed (but haven't heard back) or you've said you're not
> going to do the thing (such as move README to markdown).

Thanks. I reopened two of them. :)

>     My objection to ASAN is I'm not yet convinced it ISN'T a false positive
>     generator, although I should give it a closer look. (My first encounter with it
>     being commit 472599b99bec is a contributing factor here.)
> 
>     > with one of your diff.c changes. i've sent a patch (and a separate patch
>     to add
>     > that -Werror= to the default toybox configure, since that's one we always have
>     > to fix in the end anyway; may as well catch them fresh?).
> 
>     I agree I should hit the false positives before you hit the false positives.
> 
>     I need something like a ./testy.sh script that builds with the NDK (ASAN
>     enabled) and runs the test suite... which involves getting the test suite to
>     pass when built with the NDK. Working on it, I'll try to go faster and see if I
>     can reshuffle the priorities a bit. I have been accused of trying to boil the
>     ocean on more than one occasion...
> 
> yeah, though i think asan by default is a great idea, i think the NDK only adds
> to your problems with no real benefit.

I added ASAN=1 to the llwrap script I did building with NDK, but as I mentioned
in the github comment I need to fix passwd.c now the compile time probe isn't
skipping it. (There's no reason to use the "shadow" functions if I've got direct
/etc/passwd reading logic in lib anyway, I just hadn't finished converting stuff
over. I've got a large patch in a tree here for that, and promoting a half dozen
toys/pending/useradd.c variants as part of it, but that's DEFINITELY something
to test under mkroot and it's queued up behind getting mkroot to run the test
suite...)

> my point with asan is that any C
> programmer should probably just run with that on all the time.

This promise was made by like 15 different address sanitizer packages over the
years, from "electric fence" through the bounds checker built into tinycc, and I
broke every one I tried. But if this one's different...

(I still have "set up valgrind" on my todo list, and need a memory leak detector
for toysh when that's ready to promote. That's a long running thing where I need
to see if I've missed any allocation lifetimes after running real world scripts
through it. I can police filehandles via /proc/self/fds but not heap objects...)

> it's great at
> catching memory errors early and it's pretty damn cheap on x86-64 --- unless i'm
> benchmarking i run most stuff under asan most of the time. (and toybox doesn't
> have a lot of Android-only code, so it's unlikely that we'd get significantly
> better coverage from building with the NDK, and the *platform* is always ahead
> of the NDK anyway, so "builds with the NDK" isn't as strong a guarantee as you
> might think, even ignoring the fact that if you _do_ build with the NDK, we
> don't actually build the Android-specific stuff in exactly the same way.)

Oh sure. I've just never managed to build with the AOSP toolchain outside of
AOSP, so the NDK is the closest thing I've got. (The only musl+llvm toolchain I
have is that hexagon cross compiler...)

Alas, AOSP hasn't got an obvious mkroot equivalent where I can get JUST build a
quick boot to shell prompt test. It's "set that thing running and go to lunch"
territory, and not something you can usefully do on battery either...

Rob