[Toybox] GitHub Action Example

[enh]

On Mon, Jun 15, 2020 at 4:05 PM Rob Landley <rob at landley.net> wrote:
>
> On 6/15/20 1:42 PM, enh wrote:
> > On Mon, Jun 15, 2020 at 11:05 AM Rob Landley <rob at landley.net> wrote:
> >>> i don't actually remember us ever having an aarch64-specific issue.
> >>> (funnily enough, a 32-bit x86 build would probably find more bugs,
> >>> since i don't think anyone regularly tests any 32-bit arch locally. i
> >>> certainly only find 32-bit issues when i try to run on an Android
> >>> emulator!)
> >>
> >> I test it before releases, and I test the j-core stuff which is still only 32
> >> bit. But it's not tested nearly as regularly as the 64 bit stuff is.
> >
> > (to be clear, i meant "at the time of commit". thanks to the 32-bit
> > x86 "cuttlefish" emulator testing, we do get testing every time i try
> > to sync AOSP. but .)
>
> This wouldn't be at time of commit either, this would be at time of push to
> github. Which lately has been a day or so after my local commit on my laptop
> because I forget. :P

as far as the other ~8 billion people on the planet are concerned,
that's the same thing :-)

> That said, you've reminded me I should poke at getting an aosp build going in an
> emulator again, so I googled for "build and run aosp under qemu" and the second
> hit was an actual android link rather than third party weirdness from adware sites:
>
>
> https://android.googlesource.com/platform/external/qemu/+/gradle_0.14.4/docs/BUILDING.TXT
>
> And that says:
>
>   cd $AOSP
>   . build/envsetup.sh
>   lunch sdk-eng
>   make -j$NUM_CORES
>   emulator
>
> Which hey, seems feasible. A repo sync in the aosp I already downloaded took 22
> minutes WITH google fiber (and that was just updating the previous download
> which I last synced earlier this month). But at least the repo version that
> still runs under python 2 hasn't stopped working yet.
>
> Did you know that "lunch" without options does _not_ list sdk-eng? (Which sounds
> like it's building the sdk and not an aosp image to run under the emulator, but
> let's at least try what it says first...)

there are lots of options not listed in there. (and i don't actually
know a way to see the full list.)

>   16:57:50 Build sandboxing disabled due to nsjail error.
>
> Reassuring.
>
> I did a make without -j and it's launching 6 parallel processes on a 4 processor
> laptop. (I should have taskset it, now I know.)
>
> It gave me a package progress indicator disturbingly like yocto's (lemme guess:
> they're copying aosp), and it made it to 185 of 186 pretty quickly, and then 185
> went for a minute and a half, and it now on 187 of 396 and ONE of us can't count.

there are many different phases. all of these small-numbered ones are
basically just setting up the build.

> Sigh. I keep googling for "intro to AOSP" and either getting youtube videos from
> 2013. The frist hit that's actually on android.com rather than meetup.com or
> books.google.com is:
>
>   https://source.android.com/setup
>
> which is an explanation of governance philosophy and the "android compatibility
> program". (At least the sdk README told me what to try running.)
>
> [  5% 4111/79676] //external/flatbuffers:flatc clang++ src/idl_gen_php.cpp [linu

...that's what the real one looks like.

> Lovely. This thing really really REALLY cannot count. And at this point it's
> gonna be eating all 4 processors through dinner.
>
> (I'd kill it and restart it with taskset, but I'm not sure how to "make clean"
> and I am that guy who hits every weird dependency bug from incomplete partial
> builds pretty much every time...)

rm -rf out/

> So that's happening. Moving on...
>
> >> I'm trying to get the mkroot plumbing to run the test suite under qemu. I'm
> >> about 3/4 of the way there. That should get more variants tested in a more
> >> easily automatable fashion, but it's all musl (and glibc) unless bionic stops
> >> segfaulting "hello world" when run in a chroot that doesn't have /dev/null in it
> >> yet.
> >>
> >> But right now the test suite hasn't got nearly as much coverage as I'd like, and
> >> until I fix that running it regularly doesn't really _prove_ anything. (And
> >> finishing the shell is eating my cycles...)
> >
> > in the Dijkstra sense (page 16 of
> > http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1969.PDF), sure.
>
> In the "I add tests for the thing I just noticed was broken all the time" sense too.

i meant to also include a Dawkins "half an eye is better than no eye"
reference :-)

> > but we definitely have introduced test failures. and as we've seen
> > from Android, just running them a lot helps shake out issues too ---
> > mostly in the tests, but also in the toys. (and anyone running tests
> > downstream is going to hit test flake, so that still needs fixing.)
>
> Making the tests properly deterministic can be a challenge at times. As The
> Moment said, "I do my best."
>
> > and note also that this checks you can _build_ both glibc and musl
> > (which has been problematic in the past), both gcc and clang (which
> > has been problematic in the past), and that there are no ASan issues
> > that will prevent running the tests/toys on a HWASan/MTE aarch64
> > device.
>
> Good point. I _have_ tests for all those but they take long enough to run I tend
> to only really regression test that as part of my release process unless I
> stumble across stuff before then.
>
> The qemu stuff is intended to let me automate it so I can run it more easily and
> often, but it doesn't help with the MacOS stuff because Apple went out of their
> way to stop MacOS from running under qemu because proprietary and tied to a
> hardware dongle in a keyboard controller.

yeah, for my money the "we'll check you can still build on macos"
alone is worth it, macos being such a pain in the ass to deal with
otherwise.

> >> And 32 bit argument processing has a known structural limitation (the "truncate
> >> -s 8G" thing) which I've mentioned here before. I know how to fix it, but the
> >> fix is intrusive enough I'm not sure it's worth doing?
> >
> > (i'm much more interested in getting to where we have 64-bit-only,
> > both to replace the current 64/32 high end and the 32-only low end.)
>
> I thought Android already mostly gave up 32 bit support (all those old phones
> and tablets I can't upgrade past Marshmallow), but the embedded space ain't gonna.

L was the first 64/32 release. there has yet to be a 64-only release.
almost all devices today are 64/32, though very low-end stuff like
Android Go or watches or whatever are still 32-only.

the most recent development is that if you're a 64/32 device, you'll
get the 64-bit version of an app if the developer has both 64 and 32
versions (and if you have 32-bit native code, you're now required to
have the corresponding 64-bit code). but from toybox's perspective,
64/32 devices are already 64-only --- there's no 32-bit toybox binary
lurking there.

> Embedded still deploys plenty of _new_ 16 bit and 8 bit systems each year. Heck,
> the AVR processor in the arduino is 8-bit. (ARM is spending a lot of money to
> try to convert as much of that to Cortex-M as they can, but that's just this
> generation's "all the world's a vax".)
>
> >>>>> it seems like your setup is running on a cron-like timer? is there a way to say "on every push" instead?
> >>>>>
> >>>> There are three build triggers in the configuration, cron, push on master and pull request on master. As I forked the repo you only see changes to my repo (emolitor/toybox) trigger builds and not landley/toybox.
> >>>
> >>> ah, i see. hopefully rob will look at
> >>> https://github.com/emolitor/toybox/runs/767753680?check_suite_focus=true
> >>> and turn this on for the main repo then :-)
> >>
> >> I'm uncomfortable putting Microsoft Github dependencies directly into toybox,
> >> especially now Microsoft seems to be back on its "embrace and extend" kick:
> >>
> >>   https://www.youtube.com/watch?v=pRMb51rx2Gg#t=38s
> >
> > it's not a dependency though. just a convenience. right now, we have
> > humans doing this, and we can always go back to that if we have to.
> > but if MS is going to give you free CPU cycles to save a little bit of
> > human time...
>
> That would be the "embrace" part, yes. First one's free, don't worry it's
> harmless you won't get hooked.

i think the stated intent is to get the next generation of startups
hooked --- learn to code using github for open source stuff, why not
use the paid version when you start you company? (i'm personally
looking forward to the web editor they've talked about, if it's
anything close to Visual Studio Code --- which it ought to be, given
that that's just Javascript.)

> >> Adding a .github subdirectory to the source would be a policy change. I'm happy
> >> with a fork doing it, but am uncomfortable putting it in the main repo. (Not
> >> fatally uncomfortable, but... ergh?)
> >
> > but if it's in a fork, we don't get the benefit. that's basically back
> > to humans doing something that's a job for a computer...
>
> I "git push" from the command line and don't look at the at the web gui for days
> if not weeks at a time. What does the output of this look like? (Yet more email?)

i don't know that either. one thing i do know from other open source
projects on github is pull requests can be automatically built and
tested. that's pretty cool for both parties.

> >>>> [https://github.com/emolitor/toybox/actions/runs/134098865]
> >>>> --- a/.github/workflows/toybox.yml
> >>>> +++ b/.github/workflows/toybox.yml
> >>>> @@ -37,6 +37,20 @@ jobs:
> >>>>      - name: Test
> >>>>        run: make tests
> >>>>
> >>>> +  Ubuntu-20_04-Clang-ASAN:
> >>>> +    runs-on: ubuntu-20.04
> >>>> +
> >>>> +    steps:
> >>>> +    - uses: actions/checkout at v2
> >>>> +    - name: Setup
> >>>> +      run: sudo apt-get install build-essential clang
> >>>> +    - name: Configure
> >>>> +      run: make defconfig
> >>>> +    - name: Build
> >>>> +      run: CC=clang ASAN=1 make
> >>>> +    - name: Test
> >>>> +      run: make tests
> >>>> +
> >>
> >> Given that you aren't setting VERBOSE=fail I assume you want me to add a patch like:
> >>
> >> diff --git a/scripts/test.sh b/scripts/test.sh
> >> index 20f76d09..cdfe3bdb 100755
> >> --- a/scripts/test.sh
> >> +++ b/scripts/test.sh
> >> @@ -60,3 +60,5 @@ else
> >>      do_test "$i"
> >>    done
> >>  fi
> >> +
> >> +[ $FAILCOUNT -eq 0 ]
> >>
> >> So make can _tell_ it failed?
> >
> > (even ignoring the slightly different "run it on Android" wrapper, i
> > always run the regular toybox tests with VERBOSE=1. i don't understand
>
> I almost always run with VERBOSE=fail myself.
>
> > why i'd want to _not_ see the detail that might help me fix it.
>
> At the time I was writing it, "output turns into giant piles of noise and the
> info you want scrolled off very easily when more than one test fails".
>
> Also, tests shouldn't produce uncaptured output and it's easy to loose them in
> the noise when the output is chatty.
>
> (Keep in mind I'm the guy who made the linux kernel build output terse in the
> first place, http://lwn.net/2002/0117/a/blueberry.php3 precisely so I could see
> the warnings. Losing warnings in the noise has been a pet peeve of mine forever. :)
>
> (And yes, if I seem somewhat crotchety about the python 3 transition it's
> because I weathered the python 2 transition, as evidence in that link, and it
> DIDN'T FIX ANYTHING. C does not do this, you can still build K&R code with
> clang. Shell does not do this. This repeated flag day nonsense python keeps
> doing got old.)
>
> > especially if it's a flaky failure rather than trivially reproducible,
> > but even in the latter case it's still annoying to have to run again
> > with VERBOSE=1.)
>
> The diff makes it hard to tell _which_ test failed, and when there's more than
> one they tend to run together.

i'm not sure what you mean: not having the diff doesn't make it any easier.

> I now have VERBOSE=1,nopass,fail,xpect
>
> What the default should be is questionable, but the default probably also
> shouldn't change every time I come up with a new variant...
>
> Rob