From dahalls at gmail.com  Thu Mar 12 23:04:58 2015
From: dahalls at gmail.com (David Halls)
Date: Thu, 12 Mar 2015 23:04:58 +0000
Subject: [Aboriginal] simple-root-filesystem now lives in initramfs
	(which is actually initmpfs).
In-Reply-To: <54D66A6D.9090306@landley.net>
References: <54D66A6D.9090306@landley.net>
Message-ID: <CABX1NxF75-68H6xa4k9s3NCnwj0o468_R9ZeYpx5YRuwCt3B_Q@mail.gmail.com>

I see this is in 1.4.0 now, right? I'd still like the root fs to be
read-only so I can be sure I'm only installing into /home when building
heddle. Can I remount / as read-only? I guess I could always go back to the
old way and append the root fs into the hda sqf.
Thanks for Aboriginal, it's really cool. I managed to build right up to
Docker and Capstan and do some stuff around atomic updates using kexec.
https://github.com/davedoesdev/heddle - was about to start writing docs but
then saw 1.4.0 so that will keep me busy for a while.
Maybe you'll be interested too that I made a project to get Aboriginal
building on Travis CI:
https://github.com/davedoesdev/build-aboriginal-travis
I also used CircleCI to get public builds of heddle using Aboriginal -
artifacts here:
http://githubraw.herokuapp.com/davedoesdev/heddle/master/.circle-ci/builds.html
and build logs here: https://circleci.com/gh/davedoesdev/heddle
Probably not of interest but I thought I'd point you to it anway :-)


On 7 February 2015 at 19:41, Rob Landley <rob at landley.net> wrote:

> Largeish change I've been working on for months just went into the repo,
> it's been blocking everything else until it was done.
>
> What we _were_ doing was combining simple-root-filesystem.sh output with
> nativecompiler.sh output at build time, via the "root-filesystem.sh
> script" that basically did cp -a of the second over the first with some
> minor fixups.
>
> Now the simple-root-filesystem.sh gets packaged into a cpio.gz archive
> and fed to qemu via -initrd, and the native-compiler.sh is the /dev/hda
> and the two are spliced together at runtime by sbin/init.sh doing a
> "cp -rs /usr/hda/. /" (I.E. put symlinks in the initramfs pointing into
> the toolchain filesystem).
>
> In the new setup, the new runtime "run-emulator.sh" script doesn't feed
> in /dev/hda so you _just_ get the initramfs with simple-root-filesystem.
> Then dev-environment.sh feeds in both /dev/hda with the toolchain and
> /dev/hdb to mount on /home.
>
> This means there's no more NO_NATIVE_COMPILER control knob: if you want
> a filesystem without the native compiler run root-filesystem.sh instead
> of dev-environment.sh. I also took away ROOT_NODIRS which was a thing
> for linux from scratch chapter 5 that lfs-bootstrap.hdc never actually
> used. (It basically made aboriginal's file layout look like the /tools
> directory of linux from scratch's intermediate stage. More confusing
> than useful; now the behavior's consistent.)
>
> There's still some debris and breakage (the cp -s is saying a couple
> files conflict, both have a /bin/sh for example), and lfs-bootstrap
> build is breaking because something can't find gmp.h (no idea what's up
> with that, have to track it down). But the basics are working again.
>
> Sorry for the radio silence, that wasn't something I could do halfway
> and stop without needing to start over when I tried again...
>
> Rob
> _______________________________________________
> Aboriginal mailing list
> Aboriginal at lists.landley.net
> http://lists.landley.net/listinfo.cgi/aboriginal-landley.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20150312/fd8b83cb/attachment.htm>

From rob at landley.net  Fri Mar 13 04:17:00 2015
From: rob at landley.net (Rob Landley)
Date: Thu, 12 Mar 2015 23:17:00 -0500
Subject: [Aboriginal] simple-root-filesystem now lives in initramfs
	(which is actually initmpfs).
In-Reply-To: <CABX1NxF75-68H6xa4k9s3NCnwj0o468_R9ZeYpx5YRuwCt3B_Q@mail.gmail.com>
References: <54D66A6D.9090306@landley.net>
	<CABX1NxF75-68H6xa4k9s3NCnwj0o468_R9ZeYpx5YRuwCt3B_Q@mail.gmail.com>
Message-ID: <550264BC.5020106@landley.net>

On 03/12/2015 06:04 PM, David Halls wrote:
> I see this is in 1.4.0 now, right? I'd still like the root fs to be
> read-only so I can be sure I'm only installing into /home when building
> heddle. Can I remount / as read-only?

Yes. "mount -o remount,ro /" should work.

> I guess I could always go back to
> the old way and append the root fs into the hda sqf.
> Thanks for Aboriginal, it's really cool. I managed to build right up to
> Docker and Capstan and do some stuff around atomic updates using kexec.
> https://github.com/davedoesdev/heddle - was about to start writing docs
> but then saw 1.4.0 so that will keep me busy for a while.

I note that if you're building a nontrivial amount of stuff,
chroot-setup is probably your friend. Otherwise you'll install stuff
into / until you fill it up, it's generally only got 100 megs or so of
space. (Half of physical memory.)

> Maybe you'll be interested too that I made a project to get Aboriginal
> building on Travis
> CI: https://github.com/davedoesdev/build-aboriginal-travis

What's travis? (It sounds familiar but the pollen's screwing up my head
today...)

Ah: http://docs.travis-ci.com/user/getting-started/

Um, yay?

> I also used CircleCI to get public builds of heddle using Aboriginal -
> artifacts here:
> http://githubraw.herokuapp.com/davedoesdev/heddle/master/.circle-ci/builds.html
> and build logs here: https://circleci.com/gh/davedoesdev/heddle
> Probably not of interest but I thought I'd point you to it anway :-)

Sounds like fun.

I still have a todo item from you (grep segfault with uClibc). I should
go fix that now. (I had enough work getting the move to initramfs
working I just shipped it once it built LFS on the targets. I'm still a
kernel version behind, it's still running uclibc instead of musl...)

Rob


From dahalls at gmail.com  Fri Mar 13 06:32:06 2015
From: dahalls at gmail.com (David Halls)
Date: Fri, 13 Mar 2015 06:32:06 +0000
Subject: [Aboriginal] simple-root-filesystem now lives in initramfs
	(which is actually initmpfs).
In-Reply-To: <550264BC.5020106@landley.net>
References: <54D66A6D.9090306@landley.net>
	<CABX1NxF75-68H6xa4k9s3NCnwj0o468_R9ZeYpx5YRuwCt3B_Q@mail.gmail.com>
	<550264BC.5020106@landley.net>
Message-ID: <CABX1NxGVrMSBFkN1P9LtSRKxjGgbFe+xvQONV6RGtgUqFQP3NA@mail.gmail.com>

>
> I note that if you're building a nontrivial amount of stuff,
> chroot-setup is probably your friend. Otherwise you'll install stuff
> into / until you fill it up, it's generally only got 100 megs or so of
> space. (Half of physical memory.)
>

I'm installing everything I build into /home/install, don't touch the rest.
Then /home/install itself becomes a sqf so it can't be messed with either.


>
> I still have a todo item from you (grep segfault with uClibc). I should
> go fix that now. (I had enough work getting the move to initramfs
> working I just shipped it once it built LFS on the targets. I'm still a
> kernel version behind, it's still running uclibc instead of musl...)
>

No worries, I automate the patching anyway.
Aboriginal is good enough for me as it is, although uclibc could probably
to with some love/replacement.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20150313/4b5c0bd7/attachment.htm>

From dahalls at gmail.com  Fri Mar 13 20:13:30 2015
From: dahalls at gmail.com (David Halls)
Date: Fri, 13 Mar 2015 20:13:30 +0000
Subject: [Aboriginal] simple-root-filesystem now lives in initramfs
	(which is actually initmpfs).
In-Reply-To: <CABX1NxGVrMSBFkN1P9LtSRKxjGgbFe+xvQONV6RGtgUqFQP3NA@mail.gmail.com>
References: <54D66A6D.9090306@landley.net>
	<CABX1NxF75-68H6xa4k9s3NCnwj0o468_R9ZeYpx5YRuwCt3B_Q@mail.gmail.com>
	<550264BC.5020106@landley.net>
	<CABX1NxGVrMSBFkN1P9LtSRKxjGgbFe+xvQONV6RGtgUqFQP3NA@mail.gmail.com>
Message-ID: <CABX1NxESWSqRmHRDcHkfp6jorzv-XaKUFFmtw16GeST-h=Wc-w@mail.gmail.com>

Btw, these are the Toybox patches I'm running with:

diff -ru toybox-0.5.2.orig/lib/portability.c toybox-0.5.2/lib/portability.c
--- toybox-0.5.2.orig/lib/portability.c 2015-02-26 02:42:24.000000000 +0000
+++ toybox-0.5.2/lib/portability.c 2015-03-13 20:07:48.017855796 +0000
@@ -45,7 +45,7 @@
       new_line = realloc(*linep, new_len);
       if (!new_line) return -1;
       *np = new_len;
-      *linep = new_line;
+      line = *linep = new_line;
     }

     line[i] = ch;
@@ -59,7 +59,7 @@
     new_line = realloc(*linep, new_len);
     if (!new_line) return -1;
     *np = new_len;
-    *linep = new_line;
+    line = *linep = new_line;
   }
   line[i + 1] = '\0';

diff -ru toybox-0.5.2.orig/toys/posix/cp.c toybox-0.5.2/toys/posix/cp.c
--- toybox-0.5.2.orig/toys/posix/cp.c 2015-02-26 02:42:24.000000000 +0000
+++ toybox-0.5.2/toys/posix/cp.c 2015-03-13 20:08:18.990360352 +0000
@@ -391,6 +391,7 @@
   }

   if (toys.optflags & FLAG_D) {
+    TT.destname = toys.optargs[toys.optc-1];
     if (mkpathat(AT_FDCWD, TT.destname, 0, 2))
       perror_exit("-D '%s'", TT.destname);
     if (toys.optc == 1) return;
diff -ru toybox-0.5.2.orig/toys/posix/find.c toybox-0.5.2/toys/posix/find.c
--- toybox-0.5.2.orig/toys/posix/find.c 2015-02-26 02:42:24.000000000 +0000
+++ toybox-0.5.2/toys/posix/find.c 2015-03-13 20:07:20.927409060 +0000
@@ -435,7 +435,7 @@
           if (aa->dir && TT.topdir == -1) TT.topdir = xopen(".", 0);

         // collect names and execute commands
-        } else if (check) {
+        } else {
           char *name, *ss1 = ss[1];
           struct double_list **ddl;

@@ -443,6 +443,7 @@
           aa = (void *)llist_pop(&argdata);
           ss += aa->arglen + 1;

+          if (check) {
           // name is always a new malloc, so we can always free it.
           name = aa->dir ? xstrdup(new->name) : dirtree_path(new, 0);

@@ -479,6 +480,7 @@
           dlist_add(ddl, name);
           aa->namecount++;
           if (!aa->plus) test = flush_exec(new, aa);
+          }
         }

         // Argument consumed, skip the check.
diff -ru toybox-0.5.2.orig/toys/posix/grep.c toybox-0.5.2/toys/posix/grep.c
--- toybox-0.5.2.orig/toys/posix/grep.c 2015-02-26 02:42:24.000000000 +0000
+++ toybox-0.5.2/toys/posix/grep.c 2015-03-13 20:07:48.017855796 +0000
@@ -75,6 +75,17 @@
     return;
   }

+  struct stat stat;
+  if (fstat(fd, &stat) != 0) {
+    perror_msg("%s", name);
+    fclose(file);
+    return;
+  }
+  if (S_ISDIR(stat.st_mode)) {
+    fclose(file);
+    return;
+  }
+
   for (;;) {
     char *line = 0, *start;
     regmatch_t matches[3];


On 13 March 2015 at 06:32, David Halls <dahalls at gmail.com> wrote:

> I note that if you're building a nontrivial amount of stuff,
>> chroot-setup is probably your friend. Otherwise you'll install stuff
>> into / until you fill it up, it's generally only got 100 megs or so of
>> space. (Half of physical memory.)
>>
>
> I'm installing everything I build into /home/install, don't touch the
> rest. Then /home/install itself becomes a sqf so it can't be messed with
> either.
>
>
>>
>> I still have a todo item from you (grep segfault with uClibc). I should
>> go fix that now. (I had enough work getting the move to initramfs
>> working I just shipped it once it built LFS on the targets. I'm still a
>> kernel version behind, it's still running uclibc instead of musl...)
>>
>
> No worries, I automate the patching anyway.
> Aboriginal is good enough for me as it is, although uclibc could probably
> to with some love/replacement.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20150313/7664c6d6/attachment.htm>

From james_mcmechan at hotmail.com  Sun Mar 15 03:28:18 2015
From: james_mcmechan at hotmail.com (James McMechan)
Date: Sat, 14 Mar 2015 20:28:18 -0700
Subject: [Aboriginal] simple-root-filesystem now lives in initramfs
	(which is actually initmpfs).
In-Reply-To: <CABX1NxESWSqRmHRDcHkfp6jorzv-XaKUFFmtw16GeST-h=Wc-w@mail.gmail.com>
References: <54D66A6D.9090306@landley.net>,
	<CABX1NxF75-68H6xa4k9s3NCnwj0o468_R9ZeYpx5YRuwCt3B_Q@mail.gmail.com>,
	<550264BC.5020106@landley.net>,
	<CABX1NxGVrMSBFkN1P9LtSRKxjGgbFe+xvQONV6RGtgUqFQP3NA@mail.gmail.com>,
	<CABX1NxESWSqRmHRDcHkfp6jorzv-XaKUFFmtw16GeST-h=Wc-w@mail.gmail.com>
Message-ID: <COL125-W15B3832F78191CAE8B124DE2050@phx.gbl>

________________________________
> Date: Fri, 13 Mar 2015 20:13:30 +0000
> From: dahalls at gmail.com
> To: rob at landley.net
> CC: aboriginal at lists.landley.net
> Subject: Re: [Aboriginal] simple-root-filesystem now lives in initramfs
> (which is actually initmpfs).
>
> Btw, these are the Toybox patches I'm running with:
>
> diff -ru toybox-0.5.2.orig/lib/portability.c toybox-0.5.2/lib/portability.c
> --- toybox-0.5.2.orig/lib/portability.c 2015-02-26 02:42:24.000000000 +0000
> +++ toybox-0.5.2/lib/portability.c 2015-03-13 20:07:48.017855796 +0000
> @@ -45,7 +45,7 @@
>         new_line = realloc(*linep, new_len);
>         if (!new_line) return -1;
>         *np = new_len;
> -      *linep = new_line;
> +      line = *linep = new_line;
>       }
>
>       line[i] = ch;
> @@ -59,7 +59,7 @@
>       new_line = realloc(*linep, new_len);
>       if (!new_line) return -1;
>       *np = new_len;
> -    *linep = new_line;
> +    line = *linep = new_line;
>     }
>     line[i + 1] = '\0';
>

portability.c/getdelim under _APPLE_ & __ANDROID__ looks like it has
several glitches I find it simpler if only one alloc block can be used, instead
of three also realloc does not clear the expanded buffer so if you have
realloced and hit EOF you will end up with uninitialized data in the last byte
since it puts the \0 after where it thinks the delim is, also the 3rd realloc could
allow the \0 to write after the end of the buffer if *np == i
I can also descope two vars and eliminate one local which is nice.
For testing I ran it with the ifdef __APPLE__/__ANDROID__ commented
out.
Do you have better test cases?

The manual pages all specified that when both *linep and *np are 0 allocate
but don't mention what happens if *linep is 0 and *np is a number.
I this case I made it use *np for the first allocation.
the toybox grep does call getdelim with a var called unused
that does not get cleared between calls, would that be a bug or a feature?
My logic was that if you are calling with a number use it, the caller may
have a better guess how big it should be, and reallocating the same size
as just freed is likely best.


diff -ur toybox.old/lib/portability.c toybox/lib/portability.c
--- toybox.old/lib/portability.c??????? 2015-03-14 13:03:46.090840772 -0700
+++ toybox/lib/portability.c??? 2015-03-14 14:48:17.595678713 -0700
@@ -13,9 +13,7 @@
?ssize_t getdelim(char **linep, size_t *np, int delim, FILE *stream)
?{
?? int ch;
-? size_t new_len;
?? ssize_t i = 0;
-? char *line, *new_line;

?? // Invalid input
?? if (!linep || !np) {
@@ -23,37 +21,30 @@
???? return -1;
?? }

-? if (*linep == NULL || *np == 0) {
-??? *np = 1024;
-??? *linep = calloc(1, *np);
-??? if (*linep == NULL) return -1;
-? }
-? line = *linep;
-
-? while ((ch = getc(stream)) != EOF) {
-??? if (i> *np) {
+? while (1) {
+??? if ( *np - i < 2 || *linep == NULL) {
?????? // Need more space
-????? new_len = *np + 1024;
-????? new_line = realloc(*linep, new_len);
+????? size_t new_len = *np;
+
+????? // if we have previously allocated or new_len is too small 
+????? if (*linep || new_len < 2) new_len += 1024;
+
+????? char *new_line = realloc(*linep, new_len);
+
?????? if (!new_line) return -1;
+
?????? *np = new_len;
?????? *linep = new_line;
???? }

-??? line[i] = ch;
+??? if ((ch = getc(stream)) == EOF) break;
+
+??? (*linep)[i++] = ch;
+
???? if (ch == delim) break;
-??? i += 1;
?? }
?
-? if (i> *np) {
-??? // Need more space
-??? new_len? = i + 2;
-??? new_line = realloc(*linep, new_len);
-??? if (!new_line) return -1;
-??? *np = new_len;
-??? *linep = new_line;
-? }
-? line[i + 1] = '\0';
+? (*linep)[i] = '\0';
?
?? return i> 0 ? i : -1;
?}

> diff -ru toybox-0.5.2.orig/toys/posix/cp.c toybox-0.5.2/toys/posix/cp.c
> --- toybox-0.5.2.orig/toys/posix/cp.c 2015-02-26 02:42:24.000000000 +0000
> +++ toybox-0.5.2/toys/posix/cp.c 2015-03-13 20:08:18.990360352 +0000
> @@ -391,6 +391,7 @@
>     }
>
>     if (toys.optflags & FLAG_D) {
> +    TT.destname = toys.optargs[toys.optc-1];
>       if (mkpathat(AT_FDCWD, TT.destname, 0, 2))
>         perror_exit("-D '%s'", TT.destname);
>       if (toys.optc == 1) return;


For cp.c/do_install It is not working like the coreutils version of install
O joy, install does not respect umask in coreutils and always creates 755
toybox does respect umask, but on failure may leave the freshly
created directory behind. and while your version does not segfault it will 
not produce the target file if that optc 1: return is still present.
and even this version does not set the mode if it conflicts with the umask.
Should it just set the mode anyway or should it honor the umask?
no clue on the man install page what umask does but it does mention
the default mode of 755

diff -ur toybox.old/toys/posix/cp.c toybox/toys/posix/cp.c
--- toybox.old/toys/posix/cp.c? 2015-03-14 13:03:45.454256768 -0700
+++ toybox/toys/posix/cp.c????? 2015-03-14 16:38:07.480061072 -0700
@@ -378,9 +378,9 @@
?? }
?
?? if (toys.optflags & FLAG_D) {
+??? TT.destname = toys.optargs[toys.optc - 1];
???? if (mkpathat(AT_FDCWD, TT.destname, 0, 2))
?????? perror_exit("-D '%s'", TT.destname);
-??? if (toys.optc == 1) return;
?? }
?? if (toys.optc < 2) error_exit("needs 2 args");

$ touch from
$ chmod 777 from
$ umask 12
$ rm -rf /tmp/jim;./toybox
 install -D -m577 from /tmp/jim/mcmechan/test;ls -ld from /tmp/jim 
/tmp/jim/mcmechan /tmp/jim/mcmechan/test? 
-rwxrwxrwx 1 mcmechan mcmechan??? 0 Mar 14 16:30 from
drwxrw-r-x 3 mcmechan mcmechan 4096 Mar 14 16:30 /tmp/jim
drwxrw-r-x 2 mcmechan mcmechan 4096 Mar 14 16:30 /tmp/jim/mcmechan
-r-xrw-r-x 1 mcmechan mcmechan??? 0 Mar 14 16:30 /tmp/jim/mcmechan/test
$ rm -rf /tmp/jim;install -D -m577 from 
/tmp/jim/mcmechan/test;ls -ld from /tmp/jim /tmp/jim/mcmechan 
/tmp/jim/mcmechan/test 
-rwxrwxrwx 1 mcmechan mcmechan??? 0 Mar 14 16:30 from
drwxr-xr-x 3 mcmechan mcmechan 4096 Mar 14 16:31 /tmp/jim
drwxr-xr-x 2 mcmechan mcmechan 4096 Mar 14 16:31 /tmp/jim/mcmechan
-r-xrwxrwx 1 mcmechan mcmechan??? 0 Mar 14 16:31 /tmp/jim/mcmechan/test

$ rm -rf /tmp/jim;./toybox install -D from /tmp/jim/mcmechan/test;ls -ld from /tmp/jim /tmp/jim/mcmechan /tmp/jim/mcmechan/test 
-rwxrwxrwx 1 mcmechan mcmechan??? 0 Mar 14 16:30 from
drwxrw-r-x 3 mcmechan mcmechan 4096 Mar 14 16:33 /tmp/jim
drwxrw-r-x 2 mcmechan mcmechan 4096 Mar 14 16:33 /tmp/jim/mcmechan
-rwxrw-r-x 1 mcmechan mcmechan??? 0 Mar 14 16:33 /tmp/jim/mcmechan/test
$ rm -rf /tmp/jim;install -D from /tmp/jim/mcmechan/test;ls -ld from /tmp/jim /tmp/jim/mcmechan /tmp/jim/mcmechan/test 
-rwxrwxrwx 1 mcmechan mcmechan??? 0 Mar 14 16:30 from
drwxr-xr-x 3 mcmechan mcmechan 4096 Mar 14 16:33 /tmp/jim
drwxr-xr-x 2 mcmechan mcmechan 4096 Mar 14 16:33 /tmp/jim/mcmechan
-rwxr-xr-x 1 mcmechan mcmechan??? 0 Mar 14 16:33 /tmp/jim/mcmechan/test

$ rm -rf /tmp/jim;./toybox install -D /tmp/jim/mcmechan/test;ls -ld from /tmp/jim /tmp/jim/mcmechan /tmp/jim/mcmechan/test 
install: needs 2 args
ls: cannot access /tmp/jim/mcmechan/test: No such file or directory
-rwxrwxrwx 1 mcmechan mcmechan??? 0 Mar 14 16:30 from
drwxrw-r-x 3 mcmechan mcmechan 4096 Mar 14 16:35 /tmp/jim
drwxrw-r-x 2 mcmechan mcmechan 4096 Mar 14 16:35 /tmp/jim/mcmechan
$ rm -rf /tmp/jim;install -D /tmp/jim/mcmechan/test;ls -ld from /tmp/jim /tmp/jim/mcmechan /tmp/jim/mcmechan/test 
install: missing destination file operand after ?/tmp/jim/mcmechan/test?
Try 'install --help' for more information.
ls: cannot access /tmp/jim: No such file or directory
ls: cannot access /tmp/jim/mcmechan: No such file or directory
ls: cannot access /tmp/jim/mcmechan/test: No such file or directory
-rwxrwxrwx 1 mcmechan mcmechan 0 Mar 14 16:30 from

> diff -ru toybox-0.5.2.orig/toys/posix/find.c toybox-0.5.2/toys/posix/find.c
> --- toybox-0.5.2.orig/toys/posix/find.c 2015-02-26 02:42:24.000000000 +0000
> +++ toybox-0.5.2/toys/posix/find.c 2015-03-13 20:07:20.927409060 +0000
> @@ -435,7 +435,7 @@
>             if (aa->dir && TT.topdir == -1) TT.topdir = xopen(".", 0);
>
>           // collect names and execute commands
> -        } else if (check) {
> +        } else {
>             char *name, *ss1 = ss[1];
>             struct double_list **ddl;
>
> @@ -443,6 +443,7 @@
>             aa = (void *)llist_pop(&argdata);
>             ss += aa->arglen + 1;
>
> +          if (check) {
>             // name is always a new malloc, so we can always free it.
>             name = aa->dir ? xstrdup(new->name) : dirtree_path(new, 0);
>
> @@ -479,6 +480,7 @@
>             dlist_add(ddl, name);
>             aa->namecount++;
>             if (!aa->plus) test = flush_exec(new, aa);
> +          }
>           }
>
>           // Argument consumed, skip the check.

at the moment I don't know how you would get so check is not set on a non
first arg processing in exec ok execdir okdir?
it looks sort of like dlist_add*() is paired with llist_pop
so we are advancing the argdata pointer and ss (by the length of argdata)
what is this a fix for?

> diff -ru toybox-0.5.2.orig/toys/posix/grep.c toybox-0.5.2/toys/posix/grep.c
> --- toybox-0.5.2.orig/toys/posix/grep.c 2015-02-26 02:42:24.000000000 +0000
> +++ toybox-0.5.2/toys/posix/grep.c 2015-03-13 20:07:48.017855796 +0000
> @@ -75,6 +75,17 @@
>       return;
>     }
>
> +  struct stat stat;
> +  if (fstat(fd, &stat) != 0) {
> +    perror_msg("%s", name);
> +    fclose(file);
> +    return;
> +  }
> +  if (S_ISDIR(stat.st_mode)) {
> +    fclose(file);
> +    return;
> +  }
> +
>     for (;;) {
>       char *line = 0, *start;
>       regmatch_t matches[3];
>
For grep.c/do_grep if I move the fdopen down both fcloses can be removed
from the ifs and you then don't bother fdopening a directory just to close it.
I added setting the EISDIR and perror_msg for the IS_DIR case too like coreutils.
I am not sure how to do it correctly for socket files toybox grep got EBADFD
when coreutils reported nothing and toybox tries to go through device files when
they are recursed into
$ grep -r fred /dev/block
$ ./toybox grep -r fred /dev/block
grep: /dev/block/8:0: Bad file descriptor
?
$ grep -r fred /dev/block/8:0
grep: /dev/block/8:0: Permission denied
$ ./toybox grep -r fred /dev/block/8:0
grep: /dev/block/8:0: Bad file descriptor

diff -ur toybox.old/toys/posix/grep.c toybox/toys/posix/grep.c
--- toybox.old/toys/posix/grep.c??????? 2015-03-14 13:03:45.474254171 -0700
+++ toybox/toys/posix/grep.c??? 2015-03-14 13:17:07.270358868 -0700
@@ -52,7 +52,7 @@
?
?static void do_grep(int fd, char *name)
?{
-? FILE *file = fdopen(fd, "r");
+? FILE *file;
?? long offset = 0;
?? int lcount = 0, mcount = 0, which = toys.optflags & FLAG_w ? 2 : 0;
?? char indelim = '\n' * !(toys.optflags&FLAG_z),
@@ -60,6 +60,18 @@
?
?? if (!fd) name = "(standard input)";
?
+? struct stat stat;
+? if (fstat(fd, &stat ) != 0) {
+??? perror_msg("%s", name);
+??? return;
+??? }
+? if (S_ISDIR(stat.st_mode)) {
+??? errno=EISDIR;
+??? perror_msg("%s", name);
+??? return;
+??? }
+
+? file = fdopen(fd, "r");
?? if (!file) {
???? perror_msg("%s", name);
???? return;

>
> On 13 March 2015 at 06:32, David Halls
> <dahalls at gmail.com<mailto:dahalls at gmail.com>> wrote:
> I note that if you're building a nontrivial amount of stuff,
> chroot-setup is probably your friend. Otherwise you'll install stuff
> into / until you fill it up, it's generally only got 100 megs or so of
> space. (Half of physical memory.)
>
> I'm installing everything I build into /home/install, don't touch the
> rest. Then /home/install itself becomes a sqf so it can't be messed
> with either.
>
>
> I still have a todo item from you (grep segfault with uClibc). I should
> go fix that now. (I had enough work getting the move to initramfs
> working I just shipped it once it built LFS on the targets. I'm still a
> kernel version behind, it's still running uclibc instead of musl...)
>
> No worries, I automate the patching anyway.
> Aboriginal is good enough for me as it is, although uclibc could
> probably to with some love/replacement.
>

while running make test to double check the patches above
I found that the toybox vs coreutils chmod behave? differently
with respect to umask where it appears coreutils does not follow
the man chmod page about umask and clears the bits rather than
leaving them alone. this showed up when I accidentally left the
umask set after testing install patch, note that it says that the
umask is only used if none of [augo] are used to specify which
bits to set.

$ touch file
$ umask 124
$
 k=777;echo file set to $k;for j in "";do for i in $j+r $j-r $j=r $j+w 
$j-w $j=w $j+x $j-x $j=x; do echo;echo "testing $i";chmod $k 
file;./toybox chmod $i file;ls -l file;chmod $k file;echo; chmod $i 
file;ls -l file; done; done
file set to 777

testing +r
-rwxrwxrwx 1 mcmechan mcmechan 0 Mar 14 17:37 file

-rwxrwxrwx 1 mcmechan mcmechan 0 Mar 14 17:37 file

testing -r
--wx-wxrwx 1 mcmechan mcmechan 0 Mar 14 17:37 file

chmod: file: new permissions are -wx-wxrwx, not -wx-wx-wx
--wx-wxrwx 1 mcmechan mcmechan 0 Mar 14 17:37 file

testing =r
-r-xrw-r-- 1 mcmechan mcmechan 0 Mar 14 17:37 file

-r--r----- 1 mcmechan mcmechan 0 Mar 14 17:37 file

testing +w
-rwxrwxrwx 1 mcmechan mcmechan 0 Mar 14 17:37 file

-rwxrwxrwx 1 mcmechan mcmechan 0 Mar 14 17:37 file

testing -w
-r-xrwxr-x 1 mcmechan mcmechan 0 Mar 14 17:37 file

chmod: file: new permissions are r-xrwxr-x, not r-xr-xr-x
-r-xrwxr-x 1 mcmechan mcmechan 0 Mar 14 17:37 file

testing =w
--wx-w-rw- 1 mcmechan mcmechan 0 Mar 14 17:37 file

--w-----w- 1 mcmechan mcmechan 0 Mar 14 17:37 file

testing +x
-rwxrwxrwx 1 mcmechan mcmechan 0 Mar 14 17:37 file

-rwxrwxrwx 1 mcmechan mcmechan 0 Mar 14 17:37 file

testing -x
-rwxrw-rw- 1 mcmechan mcmechan 0 Mar 14 17:37 file

chmod: file: new permissions are rwxrw-rw-, not rw-rw-rw-
-rwxrw-rw- 1 mcmechan mcmechan 0 Mar 14 17:37 file

testing =x
---x-wxr-x 1 mcmechan mcmechan 0 Mar 14 17:37 file

------x--x 1 mcmechan mcmechan 0 Mar 14 17:37 file


by the way be sure to reset the umask before compiling

if you change k above to 000 or
change the umask to 000 they all match
because if the file has perm 000 coreutils honours the umask
for + & - r/w/x but always clears the umask for = r/w/x after
setting the bits so a file at 777 with umask 124 will end up
at 202 after chmod =w despite setting a bit that was already
set to the value in the argument
and yes I do find this somewhat insane on coreutils part
when the manual page says that the bits in umask are not
affected when u/g/o/a are not given.
 		 	   		  

From dahalls at gmail.com  Wed Mar 18 07:47:33 2015
From: dahalls at gmail.com (David Halls)
Date: Wed, 18 Mar 2015 07:47:33 +0000
Subject: [Aboriginal] simple-root-filesystem now lives in initramfs
	(which is actually initmpfs).
In-Reply-To: <COL125-W15B3832F78191CAE8B124DE2050@phx.gbl>
References: <54D66A6D.9090306@landley.net>
	<CABX1NxF75-68H6xa4k9s3NCnwj0o468_R9ZeYpx5YRuwCt3B_Q@mail.gmail.com>
	<550264BC.5020106@landley.net>
	<CABX1NxGVrMSBFkN1P9LtSRKxjGgbFe+xvQONV6RGtgUqFQP3NA@mail.gmail.com>
	<CABX1NxESWSqRmHRDcHkfp6jorzv-XaKUFFmtw16GeST-h=Wc-w@mail.gmail.com>
	<COL125-W15B3832F78191CAE8B124DE2050@phx.gbl>
Message-ID: <CABX1NxFufRg40peu53vG5h7mM9wESju4mOWuGRKHoVnYedfSdg@mail.gmail.com>

>
> portability.c/getdelim under _APPLE_ & __ANDROID__ looks like it has
> several glitches I find it simpler if only one alloc block can be used,
> instead
> of three also realloc does not clear the expanded buffer so if you have
> realloced and hit EOF you will end up with uninitialized data in the last
> byte
> since it puts the \0 after where it thinks the delim is, also the 3rd
> realloc could
> allow the \0 to write after the end of the buffer if *np == i
> I can also descope two vars and eliminate one local which is nice.
>

Do you have your changes to hand?


> For testing I ran it with the ifdef __APPLE__/__ANDROID__ commented
> out.
> Do you have better test cases?
>

I just ran grep * on a directory with subdirectories and large files.


>
> For cp.c/do_install It is not working like the coreutils version of install
> O joy, install does not respect umask in coreutils and always creates 755
> toybox does respect umask, but on failure may leave the freshly
> created directory behind. and while your version does not segfault it will
> not produce the target file if that optc 1: return is still present.
>

I do get the directory created.


>
> at the moment I don't know how you would get so check is not set on a non
> first arg processing in exec ok execdir okdir?
> it looks sort of like dlist_add*() is paired with llist_pop
> so we are advancing the argdata pointer and ss (by the length of argdata)
> what is this a fix for?
>
>
find -exec

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20150318/c9df25ab/attachment.htm>

From rob at landley.net  Thu Mar 19 21:01:23 2015
From: rob at landley.net (Rob Landley)
Date: Thu, 19 Mar 2015 16:01:23 -0500
Subject: [Aboriginal] Bug report for lfs-bootstrap
In-Reply-To: <5509F7CD.2070902@gmail.com>
References: <5500CC3C.6050804@gmail.com> <550109E3.8020703@landley.net>
	<5509F7CD.2070902@gmail.com>
Message-ID: <550B3923.8040706@landley.net>

On 03/18/2015 05:10 PM, Avery Payne wrote:
> 
> 
> On 3/11/2015 8:37 PM, Rob Landley wrote:
>>> I would like to do something that will seem a bit odd to you but makes
>>> sense for my purposes.  I would like to use Aboriginal to bootstrap a
>>> bespoke installation - basically, my own personal distro.
>> Not odd at all. This is why the build control images exist:
>>
>> http://landley.net/aboriginal/control-images
>>
>> The lfs-bootstrap one builds linux from scratch (6.8) under aboriginal.
>>
> 
> Not entirely sure why the probe is failing on but I thought I would at
> least report it.
> 
> I've had no luck in getting the current lfs-bootstrap image to build. 
> The version of grep in the current release is the culprit, with the
> message halting on a check for "support for -e and long lines" in the
> middle of a ./config or autoconfig or whatever it is.

Hmmm...

Ah! The downloads/binaries symlink pointed to 1.2.9, not 1.4.0. It was two
releases back. Yeah, that'll screw stuff up. :) (And since "bin" points
through downloads/binaries, it was going the wrong place too.)

Fixed it, let me try...

  mkdir temp
  cd temp
  wget http://landley.net/aboriginal/control-images/downloads/binaries/lfs-bootstrap.hdc
  wget http://landley.net/aboriginal/bin/system-image-i686.tar.gz
  tar xvzf system-image-i686.tar.gz
  cd system-image-i686
  ./native-build.sh ../lfs-bootstrap.hdc

There are four things it helps to have in the $PATH on the host. (Well, 5
counting the right qemu-system binary.)

1+2) toybox and busybox (it uses toybox netcat and busybox ftpd. I need
to clean up and promote toybox ftpd...)

3+4) i686-cc and distccd (to use distcc acceleration with the cross
compiler; the scripts will set it up but only if the tools are found.)

> I tried building
> some other stuff I pulled from elsewhere and also ran into the same
> "support for -e and long lines" issue.  Looking up the "-e" flag, it's
> supposed to be the expression to match against, so unless the error
> isn't stating itself correctly, that leaves the concept of "long lines"
> as the culprit for the failure, whatever the heck that means.

Leftover bug in the 1.2.9 release. Long since fixed. Sorry about that.

I sat around watching it build until it made it to coreutils (package 13
of 48, I love the title bar updating progress indicator trick). If you
still have problems, let me know.

Thanks,

Rob


From rob at landley.net  Fri Mar 27 20:20:05 2015
From: rob at landley.net (Rob Landley)
Date: Fri, 27 Mar 2015 15:20:05 -0500
Subject: [Aboriginal] grep segv in Aboriginal
In-Reply-To: <CABX1NxHyFCoVVcUk_vapu6_WsXG=hkO2erKHf1YHiq=SjM315Q@mail.gmail.com>
References: <CABX1NxHyFCoVVcUk_vapu6_WsXG=hkO2erKHf1YHiq=SjM315Q@mail.gmail.com>
Message-ID: <5515BB75.6050201@landley.net>



On 01/29/2015 01:53 AM, David Halls wrote:
> Hi-
> 
> toybox grep segvs when grepping directories but only the version
> compiled in Aboriginal:
> 
> (x86_64:1) /home # mkdir wup
> (x86_64:1) /home # toybox grep foo wup
> toybox[74]: segfault at 1003faf ip 00000000004259a4 sp 00007fff59573e30
> error 4 in toybox[400000+4e000]
> Segmentation fault
> 
> If I compile a static toybox (same version, 0.5.1) on Ubuntu (14.10) and
> copy it into Aboriginal, it doesn't segv when I use it.
> 
> Perhaps some bad optimisation or unexpected uclibc behaviour?

It's definitely a uclibc bug, doesn't happen with glibc or musl. (Or
presumably bionic.)

I have a local workaround for it (a cleaned up version of the last hunk
of the mega-patch I got sent a couple weeks ago, applied the rest to
toybox recently). I'm not checking it in to toybox might check that in
to the patches dirctory of aboriginal until I can get everything moved
over to musl.

(I spoke with Rich at ELC this week, and he took a look at getting an
llvm toolchain producing a native compiler. I believe he now understands
why I haven't already done it. :)

Rob


From dahalls at gmail.com  Sun Mar 29 06:54:52 2015
From: dahalls at gmail.com (David Halls)
Date: Sun, 29 Mar 2015 07:54:52 +0100
Subject: [Aboriginal] GCC patch licences
Message-ID: <CABX1NxHQsyGuq0DY0QdBggx-TQC=kkgcBbhBufPCYeF5Rqe7Wg@mail.gmail.com>

Hi-
I know GCC 4.2.1 is GPLv2.
Now some of the patches in Aboriginal seem to come from later GCC versions
(e.g.
http://www.crosstool-ng.org/hg/crosstool-ng/file/tip/patches/gcc/4.4.6/210-arm-unbreak-armv4t.patch
)
Does this make the GCC in Aboriginal Linux GPLv3?
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20150329/c527262f/attachment.htm>

From dahalls at gmail.com  Sun Mar 29 06:56:48 2015
From: dahalls at gmail.com (David Halls)
Date: Sun, 29 Mar 2015 07:56:48 +0100
Subject: [Aboriginal] GCC patch licences
In-Reply-To: <CABX1NxHQsyGuq0DY0QdBggx-TQC=kkgcBbhBufPCYeF5Rqe7Wg@mail.gmail.com>
References: <CABX1NxHQsyGuq0DY0QdBggx-TQC=kkgcBbhBufPCYeF5Rqe7Wg@mail.gmail.com>
Message-ID: <CABX1NxF_Db1PUqN0oRwYKM9XB3YBM4kCqGfem+=D=CLrcKAHkQ@mail.gmail.com>

Oh I see that patch is slightly different. Are there any patches from later
GCC versions in Aboriginal?

On 29 March 2015 at 07:54, David Halls <dahalls at gmail.com> wrote:

> Hi-
> I know GCC 4.2.1 is GPLv2.
> Now some of the patches in Aboriginal seem to come from later GCC versions
> (e.g.
> http://www.crosstool-ng.org/hg/crosstool-ng/file/tip/patches/gcc/4.4.6/210-arm-unbreak-armv4t.patch
> )
> Does this make the GCC in Aboriginal Linux GPLv3?
> David
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20150329/e2c9df20/attachment.htm>

From rob at landley.net  Sun Mar 29 16:10:22 2015
From: rob at landley.net (Rob Landley)
Date: Sun, 29 Mar 2015 11:10:22 -0500
Subject: [Aboriginal] GCC patch licences
In-Reply-To: <CABX1NxF_Db1PUqN0oRwYKM9XB3YBM4kCqGfem+=D=CLrcKAHkQ@mail.gmail.com>
References: <CABX1NxHQsyGuq0DY0QdBggx-TQC=kkgcBbhBufPCYeF5Rqe7Wg@mail.gmail.com>
	<CABX1NxF_Db1PUqN0oRwYKM9XB3YBM4kCqGfem+=D=CLrcKAHkQ@mail.gmail.com>
Message-ID: <551823EE.4060703@landley.net>

On 03/29/2015 01:56 AM, David Halls wrote:
> Oh I see that patch is slightly different. Are there any patches from
> later GCC versions in Aboriginal?
> 
> On 29 March 2015 at 07:54, David Halls <dahalls at gmail.com
> <mailto:dahalls at gmail.com>> wrote:
> 
>     Hi-
>     I know GCC 4.2.1 is GPLv2. 
>     Now some of the patches in Aboriginal seem to come from later GCC
>     versions (e.g.
>     http://www.crosstool-ng.org/hg/crosstool-ng/file/tip/patches/gcc/4.4.6/210-arm-unbreak-armv4t.patch)
>     Does this make the GCC in Aboriginal Linux GPLv3?
>     David

Nope, they're me independently fixing the same issues. The patch looks
similiar because that's the obvious fix, but I do _not_ take any code
from GPLv3 anyway. Ever. For any reason.

In fact, I probably derailed uclibc++ development because of this:

  http://lists.uclibc.org/pipermail/uclibc/2011-January/044697.html

At the moment I'm really hoping llvm becomes usable soon, possibly by
extracting the toolchain AOSP builds. I poked at ellcc.org but the fact
its developer extracts every upstream source tarball into his subversion
repository and then patches on top of the resulting mess makes that
really hard to work with. And when I tested his cross compiler binaries,
they were calling the host ld binary.

Rob