[Toybox] [PATCH 1/2] macOS: use getconf rather than nproc for portability.

[enh]

On Wed, Nov 28, 2018 at 2:42 PM Rob Landley <rob at landley.net> wrote:
>
> On 11/28/18 2:16 PM, enh via Toybox wrote:
> > Needed to build for macOS.
> > ---
> >  scripts/make.sh | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/scripts/make.sh b/scripts/make.sh
> > index 306a7cd..59b1fd2 100755
> > --- a/scripts/make.sh
> > +++ b/scripts/make.sh
> > @@ -15,7 +15,7 @@ source ./configure
> >  UNSTRIPPED="generated/unstripped/$(basename "$OUTNAME")"
> >
> >  # Try to keep one more cc invocation going than we have processors
> > -[ -z "$CPUS" ] && CPUS=$(($(nproc)+1))
> > +[ -z "$CPUS" ] && CPUS=$(($(getconf _NPROCESSORS_ONLN)+1))
>
> Hmmm, I did that because:
>
>   $ taskset 1 nproc
>   1
>   $ taskset 1 getconf _NPROCESSORS_ONLN
>   8
>
> Is there another...
>
>   $ taskset 1 getconf -a | grep NPROC
>   _NPROCESSORS_CONF                  8
>   _NPROCESSORS_ONLN                  8
>
> Sigh.
>
> The theory was if nproc doesn't exist then $(nproc) resolves to an empty string
> and $((+1)) becomes 1 (not an error, I checked), so it should still work, just
> single-threaded? And then you could "CPUS=1 make" to set it manually if necessary...
>
> Is this an _optimization_ for macos, or is it a build break? (I used to have a
> mac laptop but sent it to a college student on twitter who wore out her computer.)

if you're happy with just silently using one core on the mac, you
could just 2>/dev/null to avoid the error messages:

diff --git a/scripts/make.sh b/scripts/make.sh
index 306a7cd..56de2e1 100755
--- a/scripts/make.sh
+++ b/scripts/make.sh
@@ -15,7 +15,7 @@ source ./configure
 UNSTRIPPED="generated/unstripped/$(basename "$OUTNAME")"

 # Try to keep one more cc invocation going than we have processors
-[ -z "$CPUS" ] && CPUS=$(($(nproc)+1))
+[ -z "$CPUS" ] && CPUS=$(($(nproc 2>/dev/null)+1))

 if [ -z "$SED" ]
 then


> Would this work instead?
>
> if [ -z "$CPUS" ]
> then
>   # nproc respects taskset, fall back to getconf for macos
>   [ ! -z "$(type -P nproc)" ] && CPUS=$(($(nproc)+1)) ||
>     CPUS=$(($(getconf _NPROCESSORS_ONLN))
> fi

yes, modulo your typo.

diff --git a/scripts/make.sh b/scripts/make.sh
index 306a7cd..69e649f 100755
--- a/scripts/make.sh
+++ b/scripts/make.sh
@@ -15,7 +15,12 @@ source ./configure
 UNSTRIPPED="generated/unstripped/$(basename "$OUTNAME")"

 # Try to keep one more cc invocation going than we have processors
-[ -z "$CPUS" ] && CPUS=$(($(nproc)+1))
+if [ -z "$CPUS" ]
+then
+  # nproc respects taskset, fall back to getconf for macos
+  [ ! -z "$(type -P nproc)" ] && CPUS=$(($(nproc)+1)) ||
+    CPUS=$(($(getconf _NPROCESSORS_ONLN)))
+fi

 if [ -z "$SED" ]
 then

worked for me.

> (I just dug up an aboriginal linux image and confirmed that type -P exists in
> bash 2.05b from the dawn of time. Now with extra shellshock! I used to use which
> but Red Hat broke it so their version prints out crap when it _doesn't_ find
> stuff. The <strike>aristocrats</strike> people behind systemd!)
>
> >  if [ -z "$SED" ]
> >  then
>
> That was put in for macos previously. I _thought_ this worked...
>
> Hmmm... Toybox has a bootstrap issue where it provides tools that can build
> toybox but sometimes has to run in environments that don't have those tools. I
> keep thinking there should be some kind of build script that builds _just_ the
> tools needed to build toybox, without requiring a configure step. Unfortunately,
> $CFLAGS and library probes and such remain fiddly enough I haven't done it yet,
> although I should be able to come up with a "no frills" hardwired build that's
> portable-ish, for _just_ the build tools? (Yeah, which tools are the build
> tools? That's mkroot territoy...)
>
> And this wouldn't help macos because the resulting nproc (calling
> sched_getaffinity()) wouldn't _run_ on macos. (Sed probably would, unless its
> libc's regex was crazy...)

yeah, sed seems to work okay. right now (with a bunch of source hacks
that i'll try to clean up and send to you) i have the following subset
of the AOSP build tools building:

basename cat chmod cmp comm cut dirname dos2unix du echo egrep false
fgrep file grep head help hostname id ln md5sum mkdir mktemp od paste
patch pwd readlink realpath rm rmdir sed setsid sha1sum sleep sort
tee timeout true uname uniq unix2dos wc whoami xargs xxd

(note that i say "building". hostname at least doesn't work yet.)

> Rob