[Aboriginal] Load limiter patch
Rob Landley
rob at landley.net
Mon Oct 28 18:00:15 PDT 2013
On 10/16/2013 01:16:58 AM, Patrick Lauer wrote:
> On 10/16/2013 10:07 AM, Rob Landley wrote:
> > When you said "load limiter" I thought you were using the "-l LOAD"
> > option of make, I.E. --load-average. But you're just capping CPUs,
> and
> > the way you're doing it is an integer division:
> >
> > CPUS=$(( $REAL_CPUS / $TARGET_COUNT ))
> >
> > Means that if REAL_CPUS is 8 and TARGET_COUNT is 12, you get 0 (so
> each
> > target basically gets $REAL_CPUS). Or if REAL_CPUS is 16 and
> > TARGET_COUNT is 12 you get 1 and leave 4 processors idle...
>
> Yeah. You could add a fudge factor there so it overshoots a bit more
> ...
> without limiting you're going to run a local DoS ;)
>
> For my usecases it seems to work out quite well with a maximum of
> 2*REALCPU as upper limit, so on a 16-core box you'd limit to 32.
You can manually control it by specifying CPU=2 (it's the per-build
value).
As always, I need to write more documentation. And then find a way to
deal with the documentation (already) being too long for anybody to
read.
Maybe I should do an instructional video series? My new laptop probably
isn't so horribly overwhelmed to be a slideshow with audio dropouts
when I try that like the netbook was. :)
> Right now worst case you end up with 17 targets * N jobs paralleism
> per
> target, which is 17*3 with 32G or 17*6 with 64G.
>
> Unless you add nice+ionice by default that'll be very unpleasant :)
I've pondered having buildall.sh automatically do nice -n 20. Doing
ionice as well makes sense.
> (Actually - I started working on this patch because I got some
> spurious
> performance warnings from my server, which accidentally correlate with
> the times when you were logged in. So, I guess, maybe limiting
> resource
> use would be more awesome. Not that it's really causing problems ...
> just annoying)
I got a much more powerful laptop and have mostly been using that. I
might set up some cron jobs on the server to do nightly tests of new
kernels and stuff, if that's ok.
> Have fun,
>
> Patrick
Rob
1383008415.0
More information about the Aboriginal
mailing list