[Toybox] GitHub Action Example

[enh]

On Wed, Jun 17, 2020 at 1:46 PM Rob Landley <rob at landley.net> wrote:
>
> On 6/16/20 3:27 PM, enh wrote:
> > On Mon, Jun 15, 2020 at 4:05 PM Rob Landley <rob at landley.net> wrote:
> >>
> >> On 6/15/20 1:42 PM, enh wrote:
> >>> On Mon, Jun 15, 2020 at 11:05 AM Rob Landley <rob at landley.net> wrote:
> >>>>> i don't actually remember us ever having an aarch64-specific issue.
> >>>>> (funnily enough, a 32-bit x86 build would probably find more bugs,
> >>>>> since i don't think anyone regularly tests any 32-bit arch locally. i
> >>>>> certainly only find 32-bit issues when i try to run on an Android
> >>>>> emulator!)
> >>>>
> >>>> I test it before releases, and I test the j-core stuff which is still only 32
> >>>> bit. But it's not tested nearly as regularly as the 64 bit stuff is.
> >>>
> >>> (to be clear, i meant "at the time of commit". thanks to the 32-bit
> >>> x86 "cuttlefish" emulator testing, we do get testing every time i try
> >>> to sync AOSP. but .)
> >>
> >> This wouldn't be at time of commit either, this would be at time of push to
> >> github. Which lately has been a day or so after my local commit on my laptop
> >> because I forget. :P
> >
> > as far as the other ~8 billion people on the planet are concerned,
> > that's the same thing :-)
>
> Yes, but 99.99% of them get it through you, then a phone manufacturer, than a
> telecom vendor on a collective 18 month delay, and very few of them are the ones
> who have to root cause the issue and come up with a patch.

yeah, okay, you got me: mainly this would benefit me :-)

(i am volunteering to keep the mac build green though!)

> I admit this is probably a thing I should do, if for no other reason than I
> haven't got macos test hardware. (Unfortunately, the mac version of a raspberry
> pi appears to be $800: https://www.apple.com/shop/buy-mac/mac-mini .)

and even though i have a MBP available, it's sufficiently awful that i
only try it if i know there's a problem. and our automated Mac build
infrastructure is running on "Apple Pi"s too, so sucks. we find out
days later.

> >> Did you know that "lunch" without options does _not_ list sdk-eng? (Which sounds
> >> like it's building the sdk and not an aosp image to run under the emulator, but
> >> let's at least try what it says first...)
> >
> > there are lots of options not listed in there. (and i don't actually
> > know a way to see the full list.)
>
> I want to make a FAQ entry with "do this, this, and this, here's what the
> results mean". I expect multiple steps to involve "this runs overnight".
>
> The "download" part is: wget repo, repo init blah, repo sync (runs overnight).
>
> The "build" part is: cd $APSP && . build/envsetup.sh && lunch something && make
> (runs overnight)
>
> Then on day 3, you type "emulator" (at the envsetuped prompt) which I've never
> gotten to actually work for various reasons, most recently it said:
>
> emulator: WARNING: encryption is off
> Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank"
> queryCoreProfileSupport: swap interval not found
> emulator: ERROR: VkCommonOperations.cpp:496: Failed to create Vulkan instance.
> E0616 13:12:24.550453924   31310 socket_utils_common_posix.cc:201] check for
> SO_REUSEPORT: {"created":"@1592331144.550415611","description":"SO_REUSEPORT
> unavailable on compiling
> system","file":"/mnt/tmpfs/src/android/emu-master-dev/external/grpc/src/core/lib/iomgr/socket_utils_common_posix.cc","file_line":169}
> emulator: ERROR: AdbHostServer.cpp:102: Unable to connect to adb daemon on port:
> 5037
>
> And then pegged one CPU for an hour without ever drawing anything into the black
> rectangle.
>
> But if it DID work then presumably I'd adb shell into it somehow and work out
> how to run the test in there?

afaik, if you're not on the emulator team, you need to use a prebuilt.
it's broken more than it works.

i'd recommend cuttlefish
(https://source.android.com/setup/create/cuttlefish) instead. that's
what we use for our large scale testing. it's been a lifesaver while
WFH --- if you brick a real device, you probably can't get it into the
bootloader remotely!

seems like for real people their suggestion is basically "download an
image we built earlier"
(https://android.googlesource.com/device/google/cuttlefish/). which is
effectively what i do on a daily basis, just less automated.

> >> Lovely. This thing really really REALLY cannot count. And at this point it's
> >> gonna be eating all 4 processors through dinner.
> >>
> >> (I'd kill it and restart it with taskset, but I'm not sure how to "make clean"
> >> and I am that guy who hits every weird dependency bug from incomplete partial
> >> builds pretty much every time...)
> >
> > rm -rf out/
>
> Cool. Exactly what I was looking for.
>
> >>> in the Dijkstra sense (page 16 of
> >>> http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1969.PDF), sure.
> >>
> >> In the "I add tests for the thing I just noticed was broken all the time" sense too.
> >
> > i meant to also include a Dawkins "half an eye is better than no eye"
> > reference :-)
>
> Is there some phrase to encapsulate "this guy did interesting things when
> younger, then ossified into a loon"? Maybe a reference to Isaac Newton's Alchemy
> phase or Linus Pauling's "Vitamin C" phase? (Both of which were Waaaaaay before
> Bill Joy, Eric Raymond, J.K. Rowling, and Orson Scott Card did their swan dives.
> It's sad: I still occasionally want to link to 1990's dilbert comics that
> illustrates some point or other really well, but "Current Dude is Way Past His
> Sell-By Date" kinda blunts the impact...)
>
> (Yes I know about Death of the Author. It's adjacent, but not _quite_ the same
> thing? Sigh, another reason we need humanities departments and not just stem,
> "here's what tumblr and tvtropes have to say on this" isn't authoritative.)
>
> >> The qemu stuff is intended to let me automate it so I can run it more easily and
> >> often, but it doesn't help with the MacOS stuff because Apple went out of their
> >> way to stop MacOS from running under qemu because proprietary and tied to a
> >> hardware dongle in a keyboard controller.
> >
> > yeah, for my money the "we'll check you can still build on macos"
> > alone is worth it, macos being such a pain in the ass to deal with
> > otherwise.
>
> Indeed, but that provides regression testing without a development environment.
> If it finds a bug on macos, what do I _do_ about it?

assuming i can't subscribe to these (and i'm assuming i _can_), let me
know. it's always better to know about these things when there isn't a
fire drill going on.

> (I still have a todo item to dig into that ps library and try to do a
> portability.c to fill out at least _some_ of the ps fields, and I can come up
> with a freebsd test for that, and in THEORY going from freebsd to macos is like
> going from devuan to RHEL. Plenty of breakage but at least related species, and
> I've had bsd development environments running under kvm before, and can maybe
> poke Ed Maste to help there if I set aside time to work on it... :)

no, as far as i could from looking into this they're all significantly
different. you'll need to do it N times. (which is why i've _not_
volunteered for that. definitely don't care about the Mac that much!)

> >>>> And 32 bit argument processing has a known structural limitation (the "truncate
> >>>> -s 8G" thing) which I've mentioned here before. I know how to fix it, but the
> >>>> fix is intrusive enough I'm not sure it's worth doing?
> >>>
> >>> (i'm much more interested in getting to where we have 64-bit-only,
> >>> both to replace the current 64/32 high end and the 32-only low end.)
> >>
> >> I thought Android already mostly gave up 32 bit support (all those old phones
> >> and tablets I can't upgrade past Marshmallow), but the embedded space ain't gonna.
> >
> > L was the first 64/32 release. there has yet to be a 64-only release.
> > almost all devices today are 64/32, though very low-end stuff like
> > Android Go or watches or whatever are still 32-only.
>
> I should look at android go. Is that one of the "lunch" targets? (Unfortunately
> googling for "android lunch go" hits the fact you named a programming language
> go and wrote half your build system in it.)

in our circles, that's "golang". "go" is
https://www.android.com/versions/go-edition/.

afaik there's no AOSP lunch target for it. vendor blobs :-(

> > the most recent development is that if you're a 64/32 device, you'll
> > get the 64-bit version of an app if the developer has both 64 and 32
> > versions (and if you have 32-bit native code, you're now required to
> > have the corresponding 64-bit code). but from toybox's perspective,
> > 64/32 devices are already 64-only --- there's no 32-bit toybox binary
> > lurking there.
>
> The "make root" stuff should automate 32 bit testing, at least under musl.
>
> >>> it's not a dependency though. just a convenience. right now, we have
> >>> humans doing this, and we can always go back to that if we have to.
> >>> but if MS is going to give you free CPU cycles to save a little bit of
> >>> human time...
> >>
> >> That would be the "embrace" part, yes. First one's free, don't worry it's
> >> harmless you won't get hooked.
> >
> > i think the stated intent is to get the next generation of startups
> > hooked --- learn to code using github for open source stuff, why not
> > use the paid version when you start you company? (i'm personally
> > looking forward to the web editor they've talked about, if it's
> > anything close to Visual Studio Code --- which it ought to be, given
> > that that's just Javascript.)
>
> I'm aware they have a new CEO, but their core business has been monopoly
> leverage bundling for 40 years before that. (Right from the start with Ed
> Roberts that's what they were trying to establish, and Roberts got so badly
> screwed over by them he sold MITS in 1977 and went to med school. They got the
> IBM PC contract because they were the "standard" 8-bit BASIC, because Bill's
> mother was on the board of directors of the Red Cross with IBM's CEO, because
> Gates lied to Kildall about what time he'd set up the meeting with IBM at...)
>
> Ahem. Computer history hobby = I researched how this sausage was made at some
> length.

aye, but if it goes away, the worst that happens is we need to find
another way to do CI. and as the other guy on the thread said: in the
meantime, if nothing else you can feel good about eating their CPU
cycles :-)

> >>>> Adding a .github subdirectory to the source would be a policy change. I'm happy
> >>>> with a fork doing it, but am uncomfortable putting it in the main repo. (Not
> >>>> fatally uncomfortable, but... ergh?)
> >>>
> >>> but if it's in a fork, we don't get the benefit. that's basically back
> >>> to humans doing something that's a job for a computer...
> >>
> >> I "git push" from the command line and don't look at the at the web gui for days
> >> if not weeks at a time. What does the output of this look like? (Yet more email?)
> >
> > i don't know that either. one thing i do know from other open source
> > projects on github is pull requests can be automatically built and
> > tested. that's pretty cool for both parties.
>
> Yeah, JCI was paying for proprietary github when I did a contract there in 2018.
> (And Jira. And whatever IBM's calling Rational Rose this decade.) Every pull
> request build failed the entire time I was there, it only ever built right when
> merged into mainline. I asked an admin why once, and I think the answer involved
> Jenkins running containers in a container and the build's source pull from
> upstream repos getting rate limited so there was a local mirror on the wrong
> subnet of the intranet?
>
> It's been a while. I think that admin fixed it by disabling the pull builds so
> at least we didn't have to kill 'em to get the real build started faster. It
> went in the "not my problem" bucket and I moved on...
>
> Presumably Microsoft Github still works until all the people who worked at
> github when Microsoft acquired them quit. That was the classic pattern:
>
> https://web.archive.org/web/19991013145536/http://www.pbs.org/cringely/pulpit/pulpit19990826.html
>
> But who knows, maybe Microsoft is a different company now than it consistently
> was in its entire history before the current CEO started in 2014.
>
> Rob