[Toybox] Android binaries?

[enh]

On Sun, Aug 23, 2020 at 9:45 PM Rob Landley <rob at landley.net> wrote:

> On 8/21/20 6:36 PM, enh wrote:
> >
> >
> > On Thu, Aug 20, 2020 at 3:57 AM Rob Landley <rob at landley.net
> > <mailto:rob at landley.net>> wrote:
> >
> >     You said way back when that you were thinking of putting up
> downloadable current
> >     toybox android-built binaries somewhere. Did that ever happen?
> >
> > the "build a static toybox" part did, but the "add it to the artifacts"
> part
> > didn't. the NDK guy -- who spends 99% of his time with old devices and
> other
> > people also stuck on old devices -- liked the idea but the build guy --
> who
> > actually gets the bill for storage -- was less excited.
>
> I've commented before that AOSP shipping dynamically linked prebuilt
> binaries is
> one of those "so close, yet so far" things...
>

for the host binaries? yeah, static would certainly help some people, but
as long as we're using glibc for the host...


> >     I'm writing a "reporting bugs" FAQ entry because of the recent
> github thread.
> >
> >     I've also had a todo item to salvage todo entries I wrote for
> busybox forever
> >     ago, especially since the busybox devs crapped all over the current
> versions.
> >     For example,
> >
> https://git.busybox.net/busybox/tree/docs/busybox.net/FAQ.html?h=95718b309169#n361
> >     used to be project-agnostic (usable advice no matter what open
> source project
> >     you were talking about), but in current
> https://busybox.net/FAQ.html#backporting
> >     they've inserted a large digression in the middle about configuring
> busybox
> >     source from the command line to make sure it doesn't apply to any
> other project
> >     and can't be generally referenced as advice by other projects.
> >
> >     But anyway, the advice was "try to reproduce the bug on a current
> version before
> >     poking the developers because there's a nonzero chance we already
> fixed it", and
> >     for linux toybox I can point them at
> >     https://landley.net/toybox/downloads/binaries for current versions
> (even if they
> >     don't want to build it from source for their target)... but those
> are linked
> >     against musl?
> >
> >     To check if it's been fixed on _android_, they need a bionic
> version. (I mean
> >     the musl versions will run but all sorts of subtle behavior's
> different.)
> >
> > yeah, i don't know --- Android's seccomp system call filter is pretty
> narrow and
> > doesn't cover all of the "legacy" system calls that musl probably uses. i
> > suspect arm32 musl doesn't work at all, but arm64 musl might mostly work?
>
> Rich is fastidious about only using new syscalls when he can get away with
> it,
> and keeping the set of used syscalls to a minimum. I'd have to ask him
> what the
> actual set he's using is though.
>
> But mostly I was thinking that when trying to reproduce a bug, swapping
> out the
> libc probably shouldn't be part of the standard reproduction sequence.
> Even when
> the bug isn't in libc, what will/won't trigger it can change.
>
> >     And
> >     even if I build a bionic version with the NDK, that hasn't got all
> the libraries
> >     the AOSP build uses. And the AOSP build here 1) takes FOREVER,  B)
> is random git
> >     snapshot du jour, bundling one with MY releases doesn't sync up with
> YOUR
> >     releases in any useful way and could be broken because of transient
> fluff du
> >     jour, C) there's like 8 api levels for various previous releases
> still in use
> >     that I have no idea how to beat out of current AOSP source anyway,
> D) me
> >     distributing android binaries seems like a layering violation
> somehow. I do not
> >     have the domain expertise to properly support or secure them beyond
> what I'm
> >     already doing.
> >
> > the long term fix is just to get toybox into a mainline module along
> with libc.
> > but that's not happening this year or next.
> >
> > i'll see if i can work out how to tell ci.android.com <
> http://ci.android.com>
> > that toybox-static32 and toybox-static64 should only be in ndk builds...
>
> I make generally enthusiastic noises, with only the vaguest idea of what
> that
> means in context. Yay, doing the thing.
>
> Rob
>
> P.S. If you're talking about space consumed by AOSP prebuilts I have
> various
> suggestions, but that seems more a political can of worms than a technical
> one?
> I don't know the context/constraints they're working under.
>

not prebuilts, _builds_. every build that the build bots build is kept.
(exactly how long depends on exactly what kind of build. obviously some are
more important than others.)


> For example "why are you shipping 2 megs of
>
> prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9/bin/x86_64-linux-android-ld.bfd
> when ld links to .gold instead" is preceded by


depending on what you're doing, you might be using bfd, gold, or lld. lld
arrived before we'd managed to get everyone over to gold. lld is good
enough for the platform at this point but the NDK still hasn't moved, and
the NDK is trickier because we don't control all the code/build systems
people are using.


> "why are you shipping gcc when
> llvm deprecated it and it was supposedly removed" which is preceded by


for kernel builds. we're part way through the multi-year
deprecation/removal:
https://android-review.googlesource.com/c/platform/prebuilts/clang/host/linux-x86/+/1405387/3/BINUTILS_KERNEL_DEPRECATION.md#5

(it has been removed from the NDK.)


> "why does
> the x86 directory have a mips directory next to it when I thought you end
> of
> lifed that platform"... The dates on the mips files are may 6 2019 and I
> downloaded this whole thing fresh in something like february...? I'm sure
> there's a reason, but dunno what it is.
>

because it's already hard enough to build an old binutils (especially on
new macOS) and the risk/reward of changing it to not build mips binaries
that aren't used anyway just isn't worth the trouble.


> If the NDK guys are interested in experiments to save space,


the OS build is the issue here, not the NDK build.


> I note that
> building an x86-64 targeted compiler as static i686 binaries linked
> against musl
> can be SMALLER than dynamically native x86-64 binaries, and sometimes runs
> faster (due to lighter cache footprint and less memory bus bandwidth for
> the
> data structures) and does not require any :i386 packages installed to run
> because static liking. I haven't tried it with llvm but I don't see why it
> wouldn't work?
>

iirc, thanks to Windows not having symlinks, having to ship N copies of all
the libraries means that compiler size is in the noise anyway :-(


> I did this crap for YEARS in aboriginal, and I still beat similar
> weirdness out
> of musl with mcm-buildall.sh, which has:
>
>   # All toolchains after the first are themselves cross compiled (so they
>   # can be statically linked against musl on the host, for binary
> portability.)
>   # static i686 binaries are basically "poor man's x32".
>   BOOTSTRAP=i686-linux-musl
>
> near the start. :)
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/toybox-landley.net/attachments/20200824/9cbf3aec/attachment.htm>