[Aboriginal] Load limiter patch

Rob Landley rob at landley.net
Mon Oct 28 18:00:15 PDT 2013


On 10/16/2013 01:16:58 AM, Patrick Lauer wrote:
> On 10/16/2013 10:07 AM, Rob Landley wrote:
> > When you said "load limiter" I thought you were using the "-l LOAD"
> > option of make, I.E. --load-average. But you're just capping CPUs,  
> and
> > the way you're doing it is an integer division:
> >
> >   CPUS=$(( $REAL_CPUS / $TARGET_COUNT ))
> >
> > Means that if REAL_CPUS is 8 and TARGET_COUNT is 12, you get 0 (so  
> each
> > target basically gets $REAL_CPUS). Or if REAL_CPUS is 16 and
> > TARGET_COUNT is 12 you get 1 and leave 4 processors idle...
> 
> Yeah. You could add a fudge factor there so it overshoots a bit more  
> ...
>  without limiting you're going to run a local DoS ;)
> 
> For my usecases it seems to work out quite well with a maximum of
> 2*REALCPU as upper limit, so on a 16-core box you'd limit to 32.

You can manually control it by specifying CPU=2 (it's the per-build  
value).

As always, I need to write more documentation. And then find a way to  
deal with the documentation (already) being too long for anybody to  
read.

Maybe I should do an instructional video series? My new laptop probably  
isn't so horribly overwhelmed to be a slideshow with audio dropouts  
when I try that like the netbook was. :)

> Right now worst case you end up with 17 targets * N jobs paralleism  
> per
> target, which is 17*3 with 32G or 17*6 with 64G.
> 
> Unless you add nice+ionice by default that'll be very unpleasant :)

I've pondered having buildall.sh automatically do nice -n 20. Doing  
ionice as well makes sense.

> (Actually - I started working on this patch because I got some  
> spurious
> performance warnings from my server, which accidentally correlate with
> the times when you were logged in. So, I guess, maybe limiting  
> resource
> use would be more awesome. Not that it's really causing problems ...
> just annoying)

I got a much more powerful laptop and have mostly been using that. I  
might set up some cron jobs on the server to do nightly tests of new  
kernels and stuff, if that's ok.

> Have fun,
> 
> Patrick

Rob
 1383008415.0


More information about the Aboriginal mailing list