From rob at landley.net  Fri Jul  1 12:50:20 2011
From: rob at landley.net (Rob Landley)
Date: Fri, 01 Jul 2011 07:50:20 -0500
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
Message-ID: <4E0DC28C.5050209@landley.net>

I've been thinking of moving sources/control-images to its own project
for a while now, and am poking at it this morning.

I downloaded the Linux From Scratch 6.8 tarball and started triaging
what I need to upgrade in lfs-bootstrap, and this brings up the whole
"should I do extensive changes in sources/control-images or should I
bite the bullet and move that directory to its own repository?"

I'm indecisive about this because control image building has some
dependencies on Aboriginal Linux.  Then again, having another project as
a build prerequisite isn't exactly unprecedented, is it?

Mark used to do Gentoo From Scratch as an overlay, which was never quite
clean (and way too easy for me to break), but now I've done proper
"mount /dev/hdc on /mnt and run /mnt/init" control image infrastructure
allowing it to be cleanly separated, more or less.  So at what point do
I bite the bullet and cleanly separate it?

Currently, building a control image uses the download/extract/patch
infrastructure from Aboriginal Linux, but I broke that stuff out into
sources/download-functions.sh so that's easier to copy over.  Might have
to keep 'em in sync, but it's at least reasonably contained now.  The
control images based on bootstrap-skeleton depend on /sbin/setup-chroot
from the Aboriginal Linux system images, but only if the root filesystem
is read only.

The _design_ is a bit dependent: the idea behind the "bootstrap" type
images themselves is to add stuff to a minimal environment until you
reach a target state, such as "gentoo" or "fedora" or "debian".  The
design here implies you start from a known minimal state, I.E. an
Aboriginal Linux system image or really, really similar.  For example:

http://landley.net/hg/aboriginal/file/1406/sources/control-images/lfs-bootstrap/mnt/build/setup.nosrc

Is more or less a series of steps from early Linux From Scratch, but
it's specifically as applied to the Aboriginal Linux system image
starting point.

Then again, look at that URL.  All those directories mean something, but
wow that's a long path...

On the Aboriginal Linux side we have the basic automated build
infrastructure which is reasonably clean (mount /dev/hdc on /mnt and run
/mnt/init, but be rigorously agnostic about what happens from there).
And we have some convenience functions (more/build-control-images.sh)
and hooks (buildall.sh runs build-control-images.sh).  Easy enough to
clean up.

The other hiccup is uploading the result (currently through the busybox
ftp server launched by the wrapper).  I've been pondering adding diod (a
v9fs server, more portable than trying to get virtfs to work on all the
targets, and not the crawling horror NFS is at the design level) and
just having an output directory the build copies stuff to.

The reason I haven't done this is that diod (like all the NFS servers)
exports a path using the name of the directory on the host.  If it's
under an arbitrary user home directory, it could be really long and have
spaces and such in it.  Haven't wanted to deal with that...

Anyway, that's the todo item du jour.  I should go poke the diod guy and
see if I can get alias support...

Rob

 1309524620.0

From bjorn.forsman at gmail.com  Fri Jul  1 15:26:44 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Fri, 1 Jul 2011 17:26:44 +0200
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
In-Reply-To: <4E0DC28C.5050209@landley.net>
References: <4E0DC28C.5050209@landley.net>
Message-ID: <BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>

On 1 July 2011 14:50, Rob Landley <rob at landley.net> wrote:
> I've been thinking of moving sources/control-images to its own project
> for a while now, and am poking at it this morning.

Although I see the point in separating the two, because it is "clean",
how about keeping them with aboriginal a little longer, for the sake
of user + developer(?) convenience?

Best regards,
Bj?rn Forsman

 1309534004.0

From rob at landley.net  Sun Jul  3 02:08:13 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 02 Jul 2011 21:08:13 -0500
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
In-Reply-To: <BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>
References: <4E0DC28C.5050209@landley.net>
	<BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>
Message-ID: <4E0FCF0D.9060304@landley.net>

On 07/01/2011 10:26 AM, Bj?rn Forsman wrote:
> On 1 July 2011 14:50, Rob Landley <rob at landley.net> wrote:
>> I've been thinking of moving sources/control-images to its own project
>> for a while now, and am poking at it this morning.
> 
> Although I see the point in separating the two, because it is "clean",
> how about keeping them with aboriginal a little longer, for the sake
> of user + developer(?) convenience?

What use case did you have in mind?

The native build infrastructure doesn't care where the control image
came from, you just point it and go.  (You can download a system image
and a control image and set them on each other without compiling
anything on your host.)

The control images don't care hugely about the base OS they're building
more packages in, they just want a minimally functional environment to
extend.

I'd actually like to encourage people to do their _own_ control images,
and provide them infrastructure to do so.  (Yay that code getting
contributed back upstream, but it's not really a requirement either...)

What advantages do you see in having them in the same source tree?

Rob

 1309658893.0

From bjorn.forsman at gmail.com  Sun Jul  3 11:52:08 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Sun, 3 Jul 2011 13:52:08 +0200
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
In-Reply-To: <4E0FCF0D.9060304@landley.net>
References: <4E0DC28C.5050209@landley.net>
	<BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>
	<4E0FCF0D.9060304@landley.net>
Message-ID: <CAEYzJUHvL0nr5wGDFxjaELnR_wbjAX6F9B1WyH6TSA3w4S8fcA@mail.gmail.com>

2011/7/3 Rob Landley <rob at landley.net>:
> On 07/01/2011 10:26 AM, Bj?rn Forsman wrote:
>> On 1 July 2011 14:50, Rob Landley <rob at landley.net> wrote:
>>> I've been thinking of moving sources/control-images to its own project
>>> for a while now, and am poking at it this morning.
>>
>> Although I see the point in separating the two, because it is "clean",
>> how about keeping them with aboriginal a little longer, for the sake
>> of user + developer(?) convenience?
>
> What use case did you have in mind?

[Disclaimer: I don't regularly use aboriginal, but I find it very
interesting and would like to start using it to bootstrap stuff.]

For users: only one download, and having control-images in the same
repo makes them "official" and can make it easier to regression test
aboriginal.

For developers I was going to say that it makes it easy to do changes
that affect the interface between the control-images and the system
image, but as you point out below, this is not an issue:

> The native build infrastructure doesn't care where the control image
> came from, you just point it and go.  (You can download a system image
> and a control image and set them on each other without compiling
> anything on your host.)

IMHO, until the control-images in aboriginal starts to cause a lot of
churn, it is probably easier for users and developers to keep them in
the same repo. Although aboriginal is supposed to be the *minimal*
build environment that can bootstrap itself, having one extra stage
that builds a minimal gentoo/lfs/debian/... is a major boon, IMHO.

Best regards,
Bj?rn Forsman

 1309693928.0

From amimjf at sky.com  Sun Jul  3 08:06:59 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Sun, 03 Jul 2011 09:06:59 +0100
Subject: [Aboriginal] qemu-m68k
Message-ID: <4E102323.30406@sky.com>

Hi

I recently stumbled across https://gitorious.org/qemu-m68k it actually 
looks like an active project. Maybe this will finally allow more 
complete m68k support in Aboriginal.


regards,

Matthew J Fletcher


 1309680419.0

From rob at landley.net  Sun Jul  3 16:29:52 2011
From: rob at landley.net (Rob Landley)
Date: Sun, 03 Jul 2011 11:29:52 -0500
Subject: [Aboriginal] Aboriginal post from amimjf@sky.com requires approval
In-Reply-To: <mailman.20348.1309680815.5660.aboriginal-landley.net@lists.landley.net>
References: <mailman.20348.1309680815.5660.aboriginal-landley.net@lists.landley.net>
Message-ID: <4E109900.10802@landley.net>

> Hi
> 
> I recently stumbled across https://gitorious.org/qemu-m68k it actually
> looks like an active project. Maybe this will finally allow more
> complete m68k support in Aboriginal.

I have a copy of that repo here, but the last time I pulled it was 2009.

Good to know it's still active, though.  I can take a poke at it after I
figure out the current status of uClibc-0.9.32.

Thanks,

Rob.

P.S. Last night I updated the unstable URL to .32 and did a
USE_UNSTABLE=uClibc more/buildall.sh and it died trying to build i686.
Yes, the x86 developers shipped a release that DID NOT COMPILE ON X86.
Sigh.  There's a fix in the uClibc repo, commit 8245f3b4638fdff2011c,
which I've grabbed and am rebuilding.  But this does NOT fill me with
confidence that the unrelated "does not build on x86_64" bug I'd hit on
-rc3 has been fixed.  Maybe I should do a perl removal pass on glibc,
except that would probably take a month...

 1309710592.0

From rob at landley.net  Sun Jul  3 17:11:43 2011
From: rob at landley.net (Rob Landley)
Date: Sun, 03 Jul 2011 12:11:43 -0500
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
In-Reply-To: <CAEYzJUHvL0nr5wGDFxjaELnR_wbjAX6F9B1WyH6TSA3w4S8fcA@mail.gmail.com>
References: <4E0DC28C.5050209@landley.net>	<BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>	<4E0FCF0D.9060304@landley.net>
	<CAEYzJUHvL0nr5wGDFxjaELnR_wbjAX6F9B1WyH6TSA3w4S8fcA@mail.gmail.com>
Message-ID: <4E10A2CF.1080502@landley.net>

On 07/03/2011 06:52 AM, Bj?rn Forsman wrote:
> 2011/7/3 Rob Landley <rob at landley.net>:
>> On 07/01/2011 10:26 AM, Bj?rn Forsman wrote:
>>> On 1 July 2011 14:50, Rob Landley <rob at landley.net> wrote:
>>>> I've been thinking of moving sources/control-images to its own project
>>>> for a while now, and am poking at it this morning.
>>>
>>> Although I see the point in separating the two, because it is "clean",
>>> how about keeping them with aboriginal a little longer, for the sake
>>> of user + developer(?) convenience?
>>
>> What use case did you have in mind?
> 
> [Disclaimer: I don't regularly use aboriginal, but I find it very
> interesting and would like to start using it to bootstrap stuff.]
> 
> For users: only one download, and having control-images in the same
> repo makes them "official" and can make it easier to regression test
> aboriginal.

I'm all for having an official collection, but needing to get them
upstream into the aboriginal repo isn't necessarrily the best way to do
that.

Aboriginal Linux and the control images do different things.  The first
provides a generic build environment, the second builds things in that
environment.  The first project is essentially feature complete, and at
this point the changes to it are adding support for more targets and
upgrading the package versions, but not really adding a whole lot of new
_features_.  (Maybe v9fs/virtio server support to talk to the host
without an ftp server, or rewriting distcc with something simplaer and
more reliable and controllable.  But that's "doing what we already do a
better way" rather than really doing something new.)

The control images are just starting.  I have no idea what people will
make control images to _do_, and what I want to get them to do is a very
long unfinished list: gentoo support's maybe 1/3 done, LFS needs and
upgrade and to work on all targets, I need to add bootstraps for fedora
and debian and slackware and arch and who knows what else, I need to
simplify and cleanup the bootstrap-skeleton stuff...

One of the most important things about project design is knowing what
your boundaries are.  You can't answer "what does this project do"
without being able to say what it does NOT do.  Where does this project
STOP?

Aboriginal Linux's major design goal is in our motto: "we cross compile
so you don't have to".  It does all the nasty bootstrapping bits to get
you a native build environment, so you can transition to native
development.  (On target hardware, under an emulator, up to you.)
That's what we do.

What we DON'T do is be a distro.  That's the trap buildroot fell into,
and when the openembedded guys got sick of buildroot and started over
from scratch they dove right back into tha trap.  We try to keep "what
you build" and "how you build" separate.  What targets you want to
support and what packages you want to build should be separate decisions.

Because of this, we don't cross-compile more packages than absolutely
necessary.  (Even distcc is pushing it, since it _could_ be build
natively like dropbear and strace are.  But it requires host setup, so
it crosses the host/target barrier, so it can't be cleanly encapsulated
within the target in a host-agnostic way, so...)

This means the development environemnt aboriginal linux produces is as
sparse as we can make it.  You can then natively work your way up to a
full RPM or .deb or portage environment and use an existing repository
which has a full-time team maintaining it.  Or take the LFS approach of
building and installing prerequisite packages by hand.  But both of
these are nontrivial, and it's nice to provide examples of how to do it.

The problem to be solved there (by the bootstrap-skeleton stuff) is
"Fedora wants to bulid under Fedora, Debian wants to build under Debian,
Gentoo wants to build under Gentoo".  Building a Fedora root filesystem
under Gentoo is an unsolved problem which is not properly documented
anywhere.  It should be.  But that's not Aboriginal Linux's problem,
it's a separate project.

But bootstrapping distros isn't the only use of control images: you can
also regression test package du jour, with automated builds from a cron
job or similar, or running the busybox test suite (or LTP) under the
target.  Examples for how to do this are useful too, showing how to
Aboriginal Linux's "automated, under a cron job" infrastructure.

This collection of control images is unbounded.  There's no design limit
on what you put in there (not yet, anyway).  If they're part of the
Aboriginal Linux repository, then there's no limit on what we can check
into that repository...

I'm not falling into the distro trap through the backdoor either. :)

> For developers I was going to say that it makes it easy to do changes
> that affect the interface between the control-images and the system
> image, but as you point out below, this is not an issue:

It is an issue, but I explicitly _want_ to keep them separated.  I want
this interface to be stable and general purpose.  To do that, it must
remain agnostic about what's going to be using it.

You can do anything in a control image.  I'm not going to pretend I can
even imagine the kind of things you can do, it's a general purpose Linux
root filesystem you boot and run stuff in.

>> The native build infrastructure doesn't care where the control image
>> came from, you just point it and go.  (You can download a system image
>> and a control image and set them on each other without compiling
>> anything on your host.)
> 
> IMHO, until the control-images in aboriginal starts to cause a lot of
> churn, it is probably easier for users and developers to keep them in
> the same repo. Although aboriginal is supposed to be the *minimal*
> build environment that can bootstrap itself, having one extra stage
> that builds a minimal gentoo/lfs/debian/... is a major boon, IMHO.

I think it's very worth doing, and it's pretty much where I'm turning my
attention.  Aboriginal Linux itself is feature complete as of 1.0.  It
does what it set out to do, it just needs package upgrades and more
targets.  Now the action moves to the control images.

But that pretty much means finishing one project and starting another.
Providing a build environment and using that build environment are
different projects.  Aboriginal Linux is _simple_ (or at least as simple
as I know how to make it given the job it's doing), and I don't want it
to stop being simple.

All I'm proposing doing is adding another entry under
http;//landley.net/hg and maybe a new web page.  I don't expect enough
traffic to justify a separate mailing list any time soon, though, that
can stay here. :)

Rob

 1309713103.0

From bjorn.forsman at gmail.com  Sun Jul  3 18:19:59 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Sun, 3 Jul 2011 20:19:59 +0200
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
In-Reply-To: <4E10A2CF.1080502@landley.net>
References: <4E0DC28C.5050209@landley.net>
	<BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>
	<4E0FCF0D.9060304@landley.net>
	<CAEYzJUHvL0nr5wGDFxjaELnR_wbjAX6F9B1WyH6TSA3w4S8fcA@mail.gmail.com>
	<4E10A2CF.1080502@landley.net>
Message-ID: <CAEYzJUHC8ShuVMotsFfymw=CYK=VNfEDwB3Jmotj=5A6pyAXCg@mail.gmail.com>

2011/7/3 Rob Landley <rob at landley.net>:

[...on aboriginal and control-images...]

All right, you've convinced me that separating out control-images from
aboriginal is not a bad thing.

Best regards,
Bj?rn Forsman

 1309717199.0

From prazzb at gmail.com  Sun Jul  3 19:01:08 2011
From: prazzb at gmail.com (prasanna balan)
Date: Mon, 4 Jul 2011 00:31:08 +0530
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
In-Reply-To: <CAEYzJUHC8ShuVMotsFfymw=CYK=VNfEDwB3Jmotj=5A6pyAXCg@mail.gmail.com>
References: <4E0DC28C.5050209@landley.net>
	<BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>
	<4E0FCF0D.9060304@landley.net>
	<CAEYzJUHvL0nr5wGDFxjaELnR_wbjAX6F9B1WyH6TSA3w4S8fcA@mail.gmail.com>
	<4E10A2CF.1080502@landley.net>
	<CAEYzJUHC8ShuVMotsFfymw=CYK=VNfEDwB3Jmotj=5A6pyAXCg@mail.gmail.com>
Message-ID: <CAELZRKaLFW7Hsd+sY9tYUfOp6C1sJBmZSyf9-sjeBvpP3wq9KQ@mail.gmail.com>

The idea of seperation is nice, both technically and logically. Two
different projects with different ideas mustn't be together....
As for the COMPLETE dependency on aboriginal, it can be worked by assuming
aboroginal scripts folder in the root directory
and writing control images scripts in accordance...
2011/7/3 Bj?rn Forsman <bjorn.forsman at gmail.com>

> 2011/7/3 Rob Landley <rob at landley.net>:
>
> [...on aboriginal and control-images...]
>
> All right, you've convinced me that separating out control-images from
> aboriginal is not a bad thing.
>
> Best regards,
> Bj?rn Forsman
> _______________________________________________
> Aboriginal mailing list
> Aboriginal at lists.landley.net
> http://lists.landley.net/listinfo.cgi/aboriginal-landley.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110704/7a1bcf2c/attachment.htm>

From rob at landley.net  Sun Jul  3 21:39:04 2011
From: rob at landley.net (Rob Landley)
Date: Sun, 03 Jul 2011 16:39:04 -0500
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
In-Reply-To: <CAELZRKaLFW7Hsd+sY9tYUfOp6C1sJBmZSyf9-sjeBvpP3wq9KQ@mail.gmail.com>
References: <4E0DC28C.5050209@landley.net>	<BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>	<4E0FCF0D.9060304@landley.net>	<CAEYzJUHvL0nr5wGDFxjaELnR_wbjAX6F9B1WyH6TSA3w4S8fcA@mail.gmail.com>	<4E10A2CF.1080502@landley.net>	<CAEYzJUHC8ShuVMotsFfymw=CYK=VNfEDwB3Jmotj=5A6pyAXCg@mail.gmail.com>
	<CAELZRKaLFW7Hsd+sY9tYUfOp6C1sJBmZSyf9-sjeBvpP3wq9KQ@mail.gmail.com>
Message-ID: <4E10E178.7070500@landley.net>

On 07/03/2011 02:01 PM, prasanna balan wrote:
> The idea of seperation is nice, both technically and logically. Two
> different projects with different ideas mustn't be together....
> As for the COMPLETE dependency on aboriginal, it can be worked by
> assuming aboroginal scripts folder in the root directory
> and writing control images scripts in accordance...

There's two layers of dependency: creating the control images, and
running the resulting control images on a target system.

Building a "blah.hdc" image involves calling download_functions.sh
(which calls sha1file, dienow, is_in_list, announce, and dotprogress out
of utility_functions.sh), and expects to run in an environment like
host-tools.sh sets up (things like "which" and "mksquashfs" which may
not be there on all hosts).

That said, download_functions.sh and utility_functions.sh are fairly
mature these days, and I split 'em out so they could be used elsewhere.
 I'm probably ok snapshotting those and copying them into the
control-images tree.  (And if they diverge, I can either keep them in
sync by hand or not particularly care.)

When I'm building the control images, I can either install mksquashfs on
my host or run them under root-filesystem-i686.  This isn't realy
building anything significant, it's just downloading, extracting,
patching, and repackaging source code.  Copy stuff into known
directories and mksquashfs the result.  Cross compiling can fail in a
zillion different ways and do so subtly, this something like three
interesting prerequisites and isn't going to be subtle if it fails.

The other set of dependencies is _running_ control images.  In theory
you can mount it on /mnt and run /mnt/init under just about any root
filesystem.  As root.  Assuming you've got build tools, and if you don't
mind it installing all sorts of crud over your host system.

Part of what these things do is document the prerequisites and setup
steps for the Fedora build and such.  "How do you turn a minimal
environment into XXX".  However, we only test extending an Aboriginal
Linux root filesystem, a _known_ minimal starting environment.  There
aren't any EXPLICIT dependencies on it (except setup-chroot, but most
other root filesystems won't be read only, and again I may just copy
that into the control image).  There's some idiosyncratic directory
layout in there and such you won't find elsewhere.  Plus, I'm not
interested in doing the defensive programming of trying to make them
either work or fail gracefully in every possible starting environment: I
want to keep them simple.  That means making some assumptions about the
environment.  Assuming the environment is minimal and not crazy is a
pretty defensible thing to expect. :)

In some ways we just expect a busybox+uClibc environment with a
toolchain, and then replace everything we want to by installing over it.
 If you get a busybox+uClibc environment some other way, it should work
assuming everything's current versions.  If you try to run a control
image under something like Red Hat 9, or Yellow Dog Linux, or an ancient
SuSE install using a 2.4 kernel... well, maybe it'll work.  I'm not
going to stop you from doing it.

That said, lots of packages out there expect bash or perl.  There's only
one perl implementation, doesn't mean that your other thing is part of
the giant hairball that is perl, just because it uses it.  Depending on
aboriginal linux as a prerequisite is not the same as being a part of
aboriginal linux.

Rob

 1309729144.0

From prazzb at gmail.com  Mon Jul  4 08:06:24 2011
From: prazzb at gmail.com (prasanna balan)
Date: Mon, 4 Jul 2011 13:36:24 +0530
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
In-Reply-To: <4E10E178.7070500@landley.net>
References: <4E0DC28C.5050209@landley.net>
	<BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>
	<4E0FCF0D.9060304@landley.net>
	<CAEYzJUHvL0nr5wGDFxjaELnR_wbjAX6F9B1WyH6TSA3w4S8fcA@mail.gmail.com>
	<4E10A2CF.1080502@landley.net>
	<CAEYzJUHC8ShuVMotsFfymw=CYK=VNfEDwB3Jmotj=5A6pyAXCg@mail.gmail.com>
	<CAELZRKaLFW7Hsd+sY9tYUfOp6C1sJBmZSyf9-sjeBvpP3wq9KQ@mail.gmail.com>
	<4E10E178.7070500@landley.net>
Message-ID: <CAELZRKZc4ocYj2-_7HDpsNyfZEDFBCkQimKJ6W0ZsyfM6qT6qg@mail.gmail.com>

Everything else'e fine nd agreed, but I did'nt like the idea of
copying the files into the tree....... A way better idea would
be to download the latest (stable/unstable) aboriginal tarball
and build it thereby.

prazz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110704/d9f8378a/attachment.htm>

From rob at landley.net  Mon Jul  4 15:51:49 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 04 Jul 2011 10:51:49 -0500
Subject: [Aboriginal] Is moving control-images to their own repository a
	good idea?
In-Reply-To: <CAELZRKZc4ocYj2-_7HDpsNyfZEDFBCkQimKJ6W0ZsyfM6qT6qg@mail.gmail.com>
References: <4E0DC28C.5050209@landley.net>	<BANLkTiktb03FGKWZLJ4sit4oouz7AaWbFw@mail.gmail.com>	<4E0FCF0D.9060304@landley.net>	<CAEYzJUHvL0nr5wGDFxjaELnR_wbjAX6F9B1WyH6TSA3w4S8fcA@mail.gmail.com>	<4E10A2CF.1080502@landley.net>	<CAEYzJUHC8ShuVMotsFfymw=CYK=VNfEDwB3Jmotj=5A6pyAXCg@mail.gmail.com>	<CAELZRKaLFW7Hsd+sY9tYUfOp6C1sJBmZSyf9-sjeBvpP3wq9KQ@mail.gmail.com>	<4E10E178.7070500@landley.net>
	<CAELZRKZc4ocYj2-_7HDpsNyfZEDFBCkQimKJ6W0ZsyfM6qT6qg@mail.gmail.com>
Message-ID: <4E11E195.6020201@landley.net>

On 07/04/2011 03:06 AM, prasanna balan wrote:
> Everything else'e fine nd agreed, but I did'nt like the idea of
> copying the files into the tree....... A way better idea would
> be to download the latest (stable/unstable) aboriginal tarball
> and build it thereby.
> 
> prazz

I'm not too fond of copying the files into the tree either, but the ones
I'm thinking of duplicating are stable and shouldn't change.

It's a bit like copying the kconfig and bloat-o-meter stuff into busybox
rather than introducing a build-time dependency on a Linux kernel source
tarball.  (Which would have broken the busybox build on several
occasions.)  The lesser of two evils, and if they fall a bit out of sync
it's not a big deal.

Rob

 1309794709.0

From rob at landley.net  Tue Jul  5 13:42:17 2011
From: rob at landley.net (Rob Landley)
Date: Tue, 05 Jul 2011 08:42:17 -0500
Subject: [Aboriginal] New control-images repo.
Message-ID: <4E1314B9.6040207@landley.net>

There's now http://landley.net/hg/control-images which pretty much does
the same thing the code in the Aboriginal Linux tree does, with some
minor cleanups.  Not very thorougly tested, haven't removed this code
from the main tree yet.  Next up is forward porting LFS to 6.8...

Rob

 1309873337.0

From rob at landley.net  Wed Jul  6 11:52:19 2011
From: rob at landley.net (Rob Landley)
Date: Wed, 06 Jul 2011 06:52:19 -0500
Subject: [Aboriginal] Fixed control-images repo.
Message-ID: <4E144C73.9060601@landley.net>

Ahem.  The ancient version of mercurial on dreamhost doesn't support
fncache repo format, so googled the issue, pulled a non-fncache version,
and replaced the image with the copy.

Should work now, sorry 'bout that. :)

Rob

 1309953139.0

From rob at landley.net  Wed Jul  6 12:16:29 2011
From: rob at landley.net (Rob Landley)
Date: Wed, 06 Jul 2011 07:16:29 -0500
Subject: [Aboriginal] Poking at uClibc 0.9.32.
Message-ID: <4E14521D.2020107@landley.net>

I added a new smoketest-report.sh recently, which produces CSV output I
can have a trivial python script turn into html from ye olde cron job.
Here's the result for buildall with .32:

Architecture,Smoketest,Control Image,Build Stage
armv4eb,FAIL,FAIL,system-image Packaging complete
armv4l,PASS,PASS,system-image Packaging complete
armv4tl,PASS,PASS,system-image Packaging complete
armv5l,PASS,PASS,system-image Packaging complete
armv6l,PASS,PASS,system-image Packaging complete
i486,PASS,PASS,system-image Packaging complete
i586,PASS,PASS,system-image Packaging complete
i686,PASS,PASS,system-image Packaging complete
m68k,FAIL,FAIL,simple-cross-compiler alt-uClibc
mips,PASS,PASS,system-image Packaging complete
mips64,FAIL,FAIL,system-image Packaging complete
mipsel,PASS,PASS,system-image Packaging complete
powerpc,FAIL,PASS,system-image Packaging complete
powerpc-440fp,FAIL,FAIL,simple-cross-compiler binutils
sh4,FAIL,FAIL,simple-cross-compiler alt-uClibc
sparc,FAIL,FAIL,simple-cross-compiler alt-uClibc
x86_64,PASS,PASS,system-image Packaging complete

The reason powerpc is "FAIL,PASS" is because of the funky serial
"overlapping input and output confuse the IRQ handler" bug.  I'm
pondering try to add netconsole support to QEMU, since it seems fairly
straightforward:

  http://sarah.thesharps.us/2009-02-22-09-00.cherry

(I could also use busybox netcat since I wrote the implementations of it
I'm using, but having an external program for this is silly when it's
presumably not that hard for qemu to have a 10.0.2.2;6666 handler for
this that does input and output from stdin/stdout...)

Rob

 1309954589.0

From prazzb at gmail.com  Thu Jul  7 18:11:39 2011
From: prazzb at gmail.com (prasanna balan)
Date: Thu, 7 Jul 2011 23:41:39 +0530
Subject: [Aboriginal] confused about control-Images
Message-ID: <CAELZRKbdo4dpfchvRLF55qTFn6ceY7vtuxTdVqEywnA-N+BCWw@mail.gmail.com>

Hello all,
             I was poking at the control images in the aboriginal
tree,particularly hello world. I just did'nt understand what to do with
it.It wrote a script and the folder was made into an image.It gives the
resulting image of 4 kib as in the website.
            If i am not wrong,those scripts must require the previous images
i.e the directories build and packages copied into root folder of new
control-images tree and a script which first unpacks the previous image i.e
mount it somewhere to access it , adds bootstrap features and repack it and
emulate under qemu.
            Just let me know about this

prazzb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110707/1c829115/attachment.htm>

From rob at landley.net  Fri Jul  8 11:33:08 2011
From: rob at landley.net (Rob Landley)
Date: Fri, 08 Jul 2011 06:33:08 -0500
Subject: [Aboriginal] confused about control-Images
In-Reply-To: <CAELZRKbdo4dpfchvRLF55qTFn6ceY7vtuxTdVqEywnA-N+BCWw@mail.gmail.com>
References: <CAELZRKbdo4dpfchvRLF55qTFn6ceY7vtuxTdVqEywnA-N+BCWw@mail.gmail.com>
Message-ID: <4E16EAF4.7040402@landley.net>

On 07/07/2011 01:11 PM, prasanna balan wrote:
> Hello all,
>              I was poking at the control images in the aboriginal
> tree,particularly hello world. I just did'nt understand what to do with
> it.It wrote a script and the folder was made into an image.It gives the
> resulting image of 4 kib as in the website.

cd system-image-armv5l
./native-build.sh /path/to/hello-world.hdc

>             If i am not wrong,those scripts must require the previous
> images i.e the directories build and packages copied into root folder of
> new control-images tree and a script which first unpacks the previous
> image i.e mount it somewhere to access it , adds bootstrap features and
> repack it and emulate under qemu.
>             Just let me know about this


The "native-build.sh" script in each system image launches qemu with a
build control image as the third drive (/dev/hdc).  The init scripts
attempt to mount /dev/hdc on /mnt, and then run /mnt/init.  If that
works, it runs the build control image.  If that doesn't work, it falls
back to a shell prompt.

This lets you plug a separate, device-independent filesystem into your
system image and have it automaticaly take over when the emulator
finishes booting.  The images I made mostly use this to build more
software in an automated fashion, but really they could do anything.
(The busybox one runs the busybox test suite, although that hasnt' been
fully converted to build in a separate tree yet.)

> prazzb

Rob

 1310124788.0

From prazzb at gmail.com  Fri Jul  8 18:37:54 2011
From: prazzb at gmail.com (prasanna balan)
Date: Sat, 9 Jul 2011 00:07:54 +0530
Subject: [Aboriginal] confused about control-Images
In-Reply-To: <4E16EAF4.7040402@landley.net>
References: <CAELZRKbdo4dpfchvRLF55qTFn6ceY7vtuxTdVqEywnA-N+BCWw@mail.gmail.com>
	<4E16EAF4.7040402@landley.net>
Message-ID: <CAELZRKYGYOa2013SJh4ryDKuRj4Kn70c2KCC_FK5hMqKRVK_Zw@mail.gmail.com>

Hello again

The "native-build.sh" script in each system image launches qemu with a
> build control image as the third drive (/dev/hdc).  The init scripts
> attempt to mount /dev/hdc on /mnt, and then run /mnt/init.  If that
> works, it runs the build control image.  If that doesn't work, it falls
> back to a shell prompt.
>

Workin exactly as u said....(Bcoz u wrote it 2 behave that way :D)


> This lets you plug a separate, device-independent filesystem into your
> system image and have it automaticaly take over when the emulator
> finishes booting.


What i felt the best is the same image runs for all the targets

The images I made mostly use this to build more
> software in an automated fashion, but really they could do anything.
>

Know your boundaries...let it be a native compiler testing image

(The busybox one runs the busybox test suite, although that hasnt' been
> fully converted to build in a separate tree yet.)
>

Keep it going....will help wen i understand the scripts completely
Thanks
prazzb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110709/4b732ed3/attachment.htm>

From rob at landley.net  Fri Jul  8 21:40:38 2011
From: rob at landley.net (Rob Landley)
Date: Fri, 08 Jul 2011 16:40:38 -0500
Subject: [Aboriginal] confused about control-Images
In-Reply-To: <CAELZRKYGYOa2013SJh4ryDKuRj4Kn70c2KCC_FK5hMqKRVK_Zw@mail.gmail.com>
References: <CAELZRKbdo4dpfchvRLF55qTFn6ceY7vtuxTdVqEywnA-N+BCWw@mail.gmail.com>	<4E16EAF4.7040402@landley.net>
	<CAELZRKYGYOa2013SJh4ryDKuRj4Kn70c2KCC_FK5hMqKRVK_Zw@mail.gmail.com>
Message-ID: <4E177956.6090909@landley.net>

On 07/08/2011 01:37 PM, prasanna balan wrote:
> Hello again
> 
>     The "native-build.sh" script in each system image launches qemu with a
>     build control image as the third drive (/dev/hdc).  The init scripts
>     attempt to mount /dev/hdc on /mnt, and then run /mnt/init.  If that
>     works, it runs the build control image.  If that doesn't work, it falls
>     back to a shell prompt.
> 
> 
> Workin exactly as u said....(Bcoz u wrote it 2 behave that way :D)

Yay!

> 
>     This lets you plug a separate, device-independent filesystem into your
>     system image and have it automaticaly take over when the emulator
>     finishes booting. 
> 
> 
> What i felt the best is the same image runs for all the targets

It was intentional. :)

I made an image to build LILO once (x86 only), and Gentoo From Scratch
needs more host tuple information than I can easily probe (alas "$(uname
-m`-unknown-linux)" won't work on armv5l with eabi, portage wants to
know "armv5l-uclibc-gnueabi" or some such.).  But in general, the images
just do native stuff and "native" is externally provided. :)

>     The images I made mostly use this to build more
>     software in an automated fashion, but really they could do anything.
> 
> 
> Know your boundaries...let it be a native compiler testing image

Yes, but my boundaries are "I have no idea what you're going to run on
the target". :)

>     (The busybox one runs the busybox test suite, although that hasnt' been
>     fully converted to build in a separate tree yet.)
> 
>  
> Keep it going....will help wen i understand the scripts completely
> Thanks
> prazzb

Cool.  I look forward to it.

Right now I'm redoing LFS to be based on the 6.8 release, and I need to
automate some chunks of BLFS too.

Gentoo From Scratch is maybe 1/3 finished.

I haven't started debian from scratch or fedora or ubuntu or Arch Linux.
 Heck, OpenEmbedded has its own package management system with a package
repository, no reason we can't bootstrap the environment to the point
where it can natively build openembedded packages...

Lots to do here.

Rob

 1310161238.0

From amimjf at sky.com  Sun Jul 10 07:02:51 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Sun, 10 Jul 2011 08:02:51 +0100
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
Message-ID: <4E194E9B.6010209@sky.com>

Hi,

I get this printed several thousand times during compile, it does not 
seem to effect the build process, i.e for various targets it runs to 
completion.

How would i go about finding what sh is trying to invoke ?

I am running Mandriva 2010.2 on x86_64


regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310281371.0

From amimjf at sky.com  Sun Jul 10 07:15:15 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Sun, 10 Jul 2011 08:15:15 +0100
Subject: [Aboriginal] Using filesystem images on real hardware
Message-ID: <4E195183.8040907@sky.com>

Hi,

I've built various filesystem images (armv6l,i486) and run them in qemu 
using the aboriginal build machinery just fine.

Presuming a given specific target board has linux port, e.g Chumby or 
PandaBoard, i guess i just have to create a new target (settings file), 
make sure the gcc parameters are correct for soft/hard float and get a 
kernel config file for settings like the following;

CONFIG_CPU_V6=y
CONFIG_MMU=y
CONFIG_ARM_THUMB=y
CONFIG_AEABI=y
CONFIG_VFP=y

# Versatile board

CONFIG_ARCH_VERSATILE_PB=y
CONFIG_PCI_LEGACY=y
CONFIG_SERIAL_NONSTANDARD=y
CONFIG_SERIAL_AMBA_PL011=y
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
CONFIG_RTC_DRV_PL031=y
CONFIG_SCSI_SYM53C8XX_2=y
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=0
CONFIG_SCSI_SYM53C8XX_MMIO=y


how did you find that just these config options need to be set, looking 
at most linux kernel config files there are about 4000 lines in it. I 
guess most of those are defaults, do you just diff a default config file 
against a board specific one and see whats different ? then make semi 
intelligent guesses about if those changes are actually target specific 
or just the personal preference of the guy who created the kernel config 
for that board ?




regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310282115.0

From onefang at gmail.com  Sun Jul 10 07:34:14 2011
From: onefang at gmail.com (David Seikel)
Date: Sun, 10 Jul 2011 17:34:14 +1000
Subject: [Aboriginal] Using filesystem images on real hardware
In-Reply-To: <4E195183.8040907@sky.com>
References: <4E195183.8040907@sky.com>
Message-ID: <20110710173414.341e4540@U64-desktop>

On Sun, 10 Jul 2011 08:15:15 +0100 Matthew J Fletcher <amimjf at sky.com>
wrote:

> how did you find that just these config options need to be set,
> looking at most linux kernel config files there are about 4000 lines
> in it. I guess most of those are defaults, do you just diff a default
> config file against a board specific one and see whats different ?
> then make semi intelligent guesses about if those changes are
> actually target specific or just the personal preference of the guy
> who created the kernel config for that board ?

That's called a miniconfig.  Have a look at 
sources/toys/miniconfig.sh for some details about those.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110710/d59fcedf/attachment.sig>

From amimjf at sky.com  Sun Jul 10 07:43:21 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Sun, 10 Jul 2011 08:43:21 +0100
Subject: [Aboriginal] kernel build problem for m68k
Message-ID: <4E195819.6070307@sky.com>

Hi,

Using aboriginal 1.0.2

----
In file included from arch/m68k/kernel/dma.c:4:
arch/m68k/kernel/dma_no.c: In function 'dma_sync_single_for_cpu':
arch/m68k/kernel/dma_no.c:44: error: implicit declaration of function 
'flush_dcache_range'
----

I dont often look in the linux kernel arch stuff, but here goes..

flush_dcache_range() is a #define in cacheflush_no.h, so it looks like 
<asm/cacheflush.h> is not including cacheflush_no.h

looking at arch/m68k/include/asm/cacheflush.h there is a big fat #ifdef 
__uClinux__ i guess thats not being fulfilled ?




regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310283801.0

From amimjf at sky.com  Sun Jul 10 14:21:00 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Sun, 10 Jul 2011 15:21:00 +0100
Subject: [Aboriginal] Using filesystem images on real hardware
In-Reply-To: <20110710173414.341e4540@U64-desktop>
References: <4E195183.8040907@sky.com> <20110710173414.341e4540@U64-desktop>
Message-ID: <4E19B54C.7070901@sky.com>

On 10/07/11 08:34, David Seikel wrote:
> On Sun, 10 Jul 2011 08:15:15 +0100 Matthew J Fletcher<amimjf at sky.com>
> wrote:
>
>> how did you find that just these config options need to be set,
>> looking at most linux kernel config files there are about 4000 lines
>> in it. I guess most of those are defaults, do you just diff a default
>> config file against a board specific one and see whats different ?
>> then make semi intelligent guesses about if those changes are
>> actually target specific or just the personal preference of the guy
>> who created the kernel config for that board ?
>
> That's called a miniconfig.  Have a look at
> sources/toys/miniconfig.sh for some details about those.
>

Ok, i also looked the comments in the script and the docs at 
http://lwn.net/Articles/160497/ but i think i am missing a bit of 
documentation on how this works.

For my board i have a patched 2.6.31 by freescale, it comes with a 
.config file

So in the LWN article it says 'create a mini.config file', so i 'touch 
mini.config'.

I then run 'make allnoconfig KCONFIG_ALLCONFIG=mini.config', my .config 
file is now 14k instead of 17k, but mini.config is still zero bytes.

Looking at the usage for miniconfig.sh "Usage: miniconfig.sh 
configfile". I do './miniconfig.sh mini.config', which gives me the 
following output

-----------------
[mfletcher at purplebox linux-2.6.31]$ ./miniconfig.sh mini.config
cmp: EOF on mini.config
Sanity test failed, normalizing starting configuration...
--- mini.config 2011-07-10 15:09:39.665898664 +0100
+++ .config     2011-07-10 15:17:31.155212675 +0100
@@ -0,0 +1,636 @@
+#
+# Automatically generated make config: don't edit
+# Linux kernel version: 2.6.31
+#
+CONFIG_64BIT=y
+# CONFIG_X86_32 is not set
+CONFIG_X86_64=y
+CONFIG_X86=y
+CONFIG_OUTPUT_FORMAT="elf64-x86-64"
+CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
+CONFIG_GENERIC_TIME=y
+CONFIG_GENERIC_CMOS_UPDATE=y
+CONFIG_CLOCKSOURCE_WATCHDOG=y
+CONFIG_GENERIC_CLOCKEVENTS=y
+CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
+CONFIG_LOCKDEP_SUPPORT=y
+CONFIG_STACKTRACE_SUPPORT=y
+CONFIG_HAVE_LATENCYTOP_SUPPORT=y
+CONFIG_FAST_CMPXCHG_LOCAL=y
+CONFIG_MMU=y
+CONFIG_ZONE_DMA=y
+CONFIG_GENERIC_ISA_DMA=y
+CONFIG_GENERIC_IOMAP=y
+CONFIG_GENERIC_BUG=y
+CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
+CONFIG_GENERIC_HWEIGHT=y
+CONFIG_ARCH_MAY_HAVE_PC_FDC=y
+CONFIG_RWSEM_GENERIC_SPINLOCK=y
+# CONFIG_RWSEM_XCHGADD_ALGORITHM is not set
+CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
+CONFIG_GENERIC_CALIBRATE_DELAY=y
+CONFIG_GENERIC_TIME_VSYSCALL=y
+CONFIG_ARCH_HAS_CPU_RELAX=y
+CONFIG_ARCH_HAS_DEFAULT_IDLE=y
+CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
+CONFIG_HAVE_SETUP_PER_CPU_AREA=y
+CONFIG_HAVE_DYNAMIC_PER_CPU_AREA=y
+# CONFIG_HAVE_CPUMASK_OF_CPU_MAP is not set
+CONFIG_ARCH_HIBERNATION_POSSIBLE=y
+CONFIG_ARCH_SUSPEND_POSSIBLE=y
+CONFIG_ZONE_DMA32=y
+CONFIG_ARCH_POPULATES_NODE_MAP=y
+CONFIG_AUDIT_ARCH=y
+CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
+CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
+CONFIG_GENERIC_HARDIRQS=y
+CONFIG_GENERIC_HARDIRQS_NO__DO_IRQ=y
+CONFIG_GENERIC_IRQ_PROBE=y
+# CONFIG_KTIME_SCALAR is not set
+CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
+CONFIG_CONSTRUCTORS=y
+
+#
+# General setup
+#
+# CONFIG_EXPERIMENTAL is not set
+CONFIG_BROKEN_ON_SMP=y
+CONFIG_INIT_ENV_ARG_LIMIT=32
+CONFIG_LOCALVERSION=""
+# CONFIG_LOCALVERSION_AUTO is not set
+CONFIG_HAVE_KERNEL_GZIP=y
+CONFIG_HAVE_KERNEL_BZIP2=y
+CONFIG_HAVE_KERNEL_LZMA=y
+CONFIG_KERNEL_GZIP=y
+# CONFIG_KERNEL_BZIP2 is not set
+# CONFIG_KERNEL_LZMA is not set
+# CONFIG_SWAP is not set
+# CONFIG_SYSVIPC is not set
+# CONFIG_BSD_PROCESS_ACCT is not set
+
+#
+# RCU Subsystem
+#
+# CONFIG_CLASSIC_RCU is not set
+CONFIG_TREE_RCU=y
+# CONFIG_PREEMPT_RCU is not set
+# CONFIG_RCU_TRACE is not set
+CONFIG_RCU_FANOUT=64
+# CONFIG_RCU_FANOUT_EXACT is not set
+# CONFIG_TREE_RCU_TRACE is not set
+# CONFIG_PREEMPT_RCU_TRACE is not set
+# CONFIG_IKCONFIG is not set
+CONFIG_LOG_BUF_SHIFT=17
+CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
+# CONFIG_CGROUPS is not set
+# CONFIG_SYSFS_DEPRECATED_V2 is not set
+# CONFIG_RELAY is not set
+CONFIG_NAMESPACES=y
+# CONFIG_UTS_NS is not set
+# CONFIG_BLK_DEV_INITRD is not set
+# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
+CONFIG_SYSCTL=y
+CONFIG_ANON_INODES=y
+# CONFIG_EMBEDDED is not set
+CONFIG_SYSCTL_SYSCALL=y
+CONFIG_KALLSYMS=y
+# CONFIG_KALLSYMS_EXTRA_PASS is not set
+CONFIG_HOTPLUG=y
+CONFIG_PRINTK=y
+CONFIG_BUG=y
+CONFIG_ELF_CORE=y
+CONFIG_PCSPKR_PLATFORM=y
+CONFIG_BASE_FULL=y
+CONFIG_FUTEX=y
+CONFIG_EPOLL=y
+CONFIG_SIGNALFD=y
+CONFIG_TIMERFD=y
+CONFIG_EVENTFD=y
+CONFIG_SHMEM=y
+CONFIG_AIO=y
+CONFIG_HAVE_PERF_COUNTERS=y
+
+#
+# Performance Counters
+#
+# CONFIG_PERF_COUNTERS is not set
+CONFIG_VM_EVENT_COUNTERS=y
+CONFIG_SLUB_DEBUG=y
+# CONFIG_STRIP_ASM_SYMS is not set
+# CONFIG_COMPAT_BRK is not set
+# CONFIG_SLAB is not set
+CONFIG_SLUB=y
+# CONFIG_SLOB is not set
+# CONFIG_PROFILING is not set
+# CONFIG_MARKERS is not set
+CONFIG_HAVE_OPROFILE=y
+CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
+CONFIG_HAVE_IOREMAP_PROT=y
+CONFIG_HAVE_KPROBES=y
+CONFIG_HAVE_KRETPROBES=y
+CONFIG_HAVE_ARCH_TRACEHOOK=y
+CONFIG_HAVE_DMA_ATTRS=y
+CONFIG_HAVE_DMA_API_DEBUG=y
+
+#
+# GCOV-based kernel profiling
+#
+# CONFIG_SLOW_WORK is not set
+# CONFIG_HAVE_GENERIC_DMA_COHERENT is not set
+CONFIG_SLABINFO=y
+CONFIG_RT_MUTEXES=y
+CONFIG_BASE_SMALL=0
+# CONFIG_MODULES is not set
+CONFIG_BLOCK=y
+# CONFIG_BLK_DEV_BSG is not set
+# CONFIG_BLK_DEV_INTEGRITY is not set
+
+#
+# IO Schedulers
+#
+CONFIG_IOSCHED_NOOP=y
+# CONFIG_IOSCHED_AS is not set
+# CONFIG_IOSCHED_DEADLINE is not set
+# CONFIG_IOSCHED_CFQ is not set
+# CONFIG_DEFAULT_AS is not set
+# CONFIG_DEFAULT_DEADLINE is not set
+# CONFIG_DEFAULT_CFQ is not set
+CONFIG_DEFAULT_NOOP=y
+CONFIG_DEFAULT_IOSCHED="noop"
+# CONFIG_FREEZER is not set
+
+#
+# Processor type and features
+#
+# CONFIG_NO_HZ is not set
+# CONFIG_HIGH_RES_TIMERS is not set
+CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
+# CONFIG_SMP is not set
+CONFIG_X86_MPPARSE=y
+# CONFIG_X86_EXTENDED_PLATFORM is not set
+# CONFIG_SCHED_OMIT_FRAME_POINTER is not set
+# CONFIG_PARAVIRT_GUEST is not set
+# CONFIG_MEMTEST is not set
+# CONFIG_M386 is not set
+# CONFIG_M486 is not set
+# CONFIG_M586 is not set
+# CONFIG_M586TSC is not set
+# CONFIG_M586MMX is not set
+# CONFIG_M686 is not set
+# CONFIG_MPENTIUMII is not set
+# CONFIG_MPENTIUMIII is not set
+# CONFIG_MPENTIUMM is not set
+# CONFIG_MPENTIUM4 is not set
+# CONFIG_MK6 is not set
+# CONFIG_MK7 is not set
+# CONFIG_MK8 is not set
+# CONFIG_MCRUSOE is not set
+# CONFIG_MEFFICEON is not set
+# CONFIG_MWINCHIPC6 is not set
+# CONFIG_MWINCHIP3D is not set
+# CONFIG_MGEODEGX1 is not set
+# CONFIG_MGEODE_LX is not set
+# CONFIG_MCYRIXIII is not set
+# CONFIG_MVIAC3_2 is not set
+# CONFIG_MVIAC7 is not set
+# CONFIG_MPSC is not set
+# CONFIG_MCORE2 is not set
+CONFIG_GENERIC_CPU=y
+CONFIG_X86_CPU=y
+CONFIG_X86_L1_CACHE_BYTES=64
+CONFIG_X86_INTERNODE_CACHE_BYTES=64
+CONFIG_X86_CMPXCHG=y
+CONFIG_X86_L1_CACHE_SHIFT=6
+CONFIG_X86_WP_WORKS_OK=y
+CONFIG_X86_TSC=y
+CONFIG_X86_CMPXCHG64=y
+CONFIG_X86_CMOV=y
+CONFIG_X86_MINIMUM_CPU_FAMILY=64
+CONFIG_X86_DEBUGCTLMSR=y
+CONFIG_CPU_SUP_INTEL=y
+CONFIG_CPU_SUP_AMD=y
+CONFIG_CPU_SUP_CENTAUR=y
+# CONFIG_X86_DS is not set
+CONFIG_HPET_TIMER=y
+CONFIG_DMI=y
+CONFIG_SWIOTLB=y
+CONFIG_IOMMU_HELPER=y
+# CONFIG_IOMMU_API is not set
+CONFIG_NR_CPUS=1
+CONFIG_PREEMPT_NONE=y
+# CONFIG_PREEMPT_VOLUNTARY is not set
+# CONFIG_PREEMPT is not set
+CONFIG_X86_LOCAL_APIC=y
+CONFIG_X86_IO_APIC=y
+# CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS is not set
+# CONFIG_X86_MCE is not set
+# CONFIG_I8K is not set
+# CONFIG_MICROCODE is not set
+# CONFIG_X86_MSR is not set
+# CONFIG_X86_CPUID is not set
+# CONFIG_X86_CPU_DEBUG is not set
+CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
+CONFIG_DIRECT_GBPAGES=y
+CONFIG_ARCH_SPARSEMEM_DEFAULT=y
+CONFIG_ARCH_SPARSEMEM_ENABLE=y
+CONFIG_ARCH_SELECT_MEMORY_MODEL=y
+CONFIG_SELECT_MEMORY_MODEL=y
+# CONFIG_FLATMEM_MANUAL is not set
+# CONFIG_DISCONTIGMEM_MANUAL is not set
+CONFIG_SPARSEMEM_MANUAL=y
+CONFIG_SPARSEMEM=y
+CONFIG_HAVE_MEMORY_PRESENT=y
+CONFIG_SPARSEMEM_EXTREME=y
+CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
+# CONFIG_SPARSEMEM_VMEMMAP is not set
+# CONFIG_MEMORY_HOTPLUG is not set
+CONFIG_PAGEFLAGS_EXTENDED=y
+CONFIG_SPLIT_PTLOCK_CPUS=4
+CONFIG_PHYS_ADDR_T_64BIT=y
+CONFIG_ZONE_DMA_FLAG=1
+CONFIG_BOUNCE=y
+CONFIG_VIRT_TO_BUS=y
+CONFIG_HAVE_MLOCK=y
+CONFIG_HAVE_MLOCKED_PAGE_BIT=y
+CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
+# CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
+# CONFIG_X86_RESERVE_LOW_64K is not set
+# CONFIG_MTRR is not set
+# CONFIG_SECCOMP is not set
+# CONFIG_CC_STACKPROTECTOR is not set
+# CONFIG_HZ_100 is not set
+CONFIG_HZ_250=y
+# CONFIG_HZ_300 is not set
+# CONFIG_HZ_1000 is not set
+CONFIG_HZ=250
+# CONFIG_SCHED_HRTICK is not set
+# CONFIG_KEXEC is not set
+# CONFIG_CRASH_DUMP is not set
+CONFIG_PHYSICAL_START=0x1000000
+# CONFIG_RELOCATABLE is not set
+CONFIG_PHYSICAL_ALIGN=0x1000000
+# CONFIG_CMDLINE_BOOL is not set
+CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
+
+#
+# Power management and ACPI options
+#
+# CONFIG_PM is not set
+
+#
+# CPU Frequency scaling
+#
+# CONFIG_CPU_FREQ is not set
+# CONFIG_CPU_IDLE is not set
+
+#
+# Memory power savings
+#
+
+#
+# Bus options (PCI etc.)
+#
+# CONFIG_PCI is not set
+# CONFIG_ARCH_SUPPORTS_MSI is not set
+CONFIG_ISA_DMA_API=y
+# CONFIG_PCCARD is not set
+
+#
+# Executable file formats / Emulations
+#
+# CONFIG_BINFMT_ELF is not set
+# CONFIG_HAVE_AOUT is not set
+# CONFIG_BINFMT_MISC is not set
+# CONFIG_IA32_EMULATION is not set
+# CONFIG_COMPAT_FOR_U64_ALIGNMENT is not set
+# CONFIG_NET is not set
+
+#
+# Device Drivers
+#
+
+#
+# Generic Driver Options
+#
+CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_STANDALONE=y
+# CONFIG_PREVENT_FIRMWARE_BUILD is not set
+CONFIG_FW_LOADER=y
+# CONFIG_FIRMWARE_IN_KERNEL is not set
+CONFIG_EXTRA_FIRMWARE=""
+# CONFIG_SYS_HYPERVISOR is not set
+# CONFIG_MTD is not set
+# CONFIG_PARPORT is not set
+# CONFIG_BLK_DEV is not set
+# CONFIG_MISC_DEVICES is not set
+CONFIG_HAVE_IDE=y
+# CONFIG_IDE is not set
+
+#
+# SCSI device support
+#
+# CONFIG_RAID_ATTRS is not set
+# CONFIG_SCSI is not set
+# CONFIG_SCSI_DMA is not set
+# CONFIG_SCSI_NETLINK is not set
+# CONFIG_ATA is not set
+# CONFIG_MD is not set
+# CONFIG_MACINTOSH_DRIVERS is not set
+# CONFIG_PHONE is not set
+
+#
+# Input device support
+#
+CONFIG_INPUT=y
+# CONFIG_INPUT_FF_MEMLESS is not set
+# CONFIG_INPUT_POLLDEV is not set
+
+#
+# Userland interfaces
+#
+CONFIG_INPUT_MOUSEDEV=y
+# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
+CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
+CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
+# CONFIG_INPUT_JOYDEV is not set
+# CONFIG_INPUT_EVDEV is not set
+# CONFIG_INPUT_EVBUG is not set
+
+#
+# Input Device Drivers
+#
+CONFIG_INPUT_KEYBOARD=y
+CONFIG_KEYBOARD_ATKBD=y
+# CONFIG_KEYBOARD_LKKBD is not set
+# CONFIG_KEYBOARD_NEWTON is not set
+# CONFIG_KEYBOARD_STOWAWAY is not set
+# CONFIG_KEYBOARD_SUNKBD is not set
+# CONFIG_KEYBOARD_XTKBD is not set
+# CONFIG_INPUT_MOUSE is not set
+# CONFIG_INPUT_JOYSTICK is not set
+# CONFIG_INPUT_TABLET is not set
+# CONFIG_INPUT_TOUCHSCREEN is not set
+# CONFIG_INPUT_MISC is not set
+
+#
+# Hardware I/O ports
+#
+CONFIG_SERIO=y
+CONFIG_SERIO_I8042=y
+# CONFIG_SERIO_SERPORT is not set
+# CONFIG_SERIO_CT82C710 is not set
+CONFIG_SERIO_LIBPS2=y
+# CONFIG_SERIO_RAW is not set
+# CONFIG_GAMEPORT is not set
+
+#
+# Character devices
+#
+CONFIG_VT=y
+CONFIG_CONSOLE_TRANSLATIONS=y
+CONFIG_VT_CONSOLE=y
+CONFIG_HW_CONSOLE=y
+# CONFIG_VT_HW_CONSOLE_BINDING is not set
+# CONFIG_DEVKMEM is not set
+# CONFIG_SERIAL_NONSTANDARD is not set
+
+#
+# Serial drivers
+#
+# CONFIG_SERIAL_8250 is not set
+CONFIG_FIX_EARLYCON_MEM=y
+
+#
+# Non-8250 serial port support
+#
+CONFIG_UNIX98_PTYS=y
+# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
+# CONFIG_LEGACY_PTYS is not set
+# CONFIG_IPMI_HANDLER is not set
+# CONFIG_HW_RANDOM is not set
+# CONFIG_NVRAM is not set
+# CONFIG_RTC is not set
+# CONFIG_GEN_RTC is not set
+# CONFIG_R3964 is not set
+# CONFIG_MWAVE is not set
+# CONFIG_PC8736x_GPIO is not set
+# CONFIG_RAW_DRIVER is not set
+# CONFIG_HANGCHECK_TIMER is not set
+# CONFIG_I2C is not set
+# CONFIG_SPI is not set
+
+#
+# PPS support
+#
+CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
+# CONFIG_GPIOLIB is not set
+# CONFIG_W1 is not set
+# CONFIG_POWER_SUPPLY is not set
+# CONFIG_HWMON is not set
+# CONFIG_THERMAL is not set
+# CONFIG_THERMAL_HWMON is not set
+# CONFIG_WATCHDOG is not set
+CONFIG_SSB_POSSIBLE=y
+
+#
+# Sonics Silicon Backplane
+#
+# CONFIG_SSB is not set
+
+#
+# Multifunction device drivers
+#
+# CONFIG_MFD_CORE is not set
+# CONFIG_MFD_SM501 is not set
+# CONFIG_HTC_PASIC3 is not set
+# CONFIG_MFD_TMIO is not set
+# CONFIG_REGULATOR is not set
+# CONFIG_MEDIA_SUPPORT is not set
+
+#
+# Graphics support
+#
+# CONFIG_VGASTATE is not set
+# CONFIG_VIDEO_OUTPUT_CONTROL is not set
+# CONFIG_FB is not set
+# CONFIG_BACKLIGHT_LCD_SUPPORT is not set
+
+#
+# Display device support
+#
+# CONFIG_DISPLAY_SUPPORT is not set
+
+#
+# Console display driver support
+#
+CONFIG_VGA_CONSOLE=y
+# CONFIG_VGACON_SOFT_SCROLLBACK is not set
+CONFIG_DUMMY_CONSOLE=y
+# CONFIG_SOUND is not set
+# CONFIG_HID_SUPPORT is not set
+# CONFIG_USB_SUPPORT is not set
+# CONFIG_MMC is not set
+# CONFIG_MEMSTICK is not set
+# CONFIG_NEW_LEDS is not set
+# CONFIG_ACCESSIBILITY is not set
+# CONFIG_EDAC is not set
+# CONFIG_RTC_CLASS is not set
+# CONFIG_DMADEVICES is not set
+# CONFIG_AUXDISPLAY is not set
+# CONFIG_UIO is not set
+
+#
+# TI VLYNQ
+#
+# CONFIG_STAGING is not set
+# CONFIG_X86_PLATFORM_DEVICES is not set
+
+#
+# Firmware Drivers
+#
+# CONFIG_EDD is not set
+CONFIG_FIRMWARE_MEMMAP=y
+# CONFIG_DELL_RBU is not set
+# CONFIG_DCDBAS is not set
+# CONFIG_DMIID is not set
+# CONFIG_ISCSI_IBFT_FIND is not set
+
+#
+# File systems
+#
+# CONFIG_EXT2_FS is not set
+# CONFIG_EXT3_FS is not set
+# CONFIG_EXT4_FS is not set
+# CONFIG_REISERFS_FS is not set
+# CONFIG_JFS_FS is not set
+# CONFIG_FS_POSIX_ACL is not set
+# CONFIG_XFS_FS is not set
+CONFIG_FILE_LOCKING=y
+# CONFIG_FSNOTIFY is not set
+# CONFIG_DNOTIFY is not set
+# CONFIG_INOTIFY is not set
+# CONFIG_INOTIFY_USER is not set
+# CONFIG_QUOTA is not set
+# CONFIG_AUTOFS_FS is not set
+# CONFIG_AUTOFS4_FS is not set
+# CONFIG_FUSE_FS is not set
+
+#
+# Caches
+#
+
+#
+# CD-ROM/DVD Filesystems
+#
+# CONFIG_ISO9660_FS is not set
+# CONFIG_UDF_FS is not set
+
+#
+# DOS/FAT/NT Filesystems
+#
+# CONFIG_MSDOS_FS is not set
+# CONFIG_VFAT_FS is not set
+# CONFIG_NTFS_FS is not set
+
+#
+# Pseudo filesystems
+#
+CONFIG_PROC_FS=y
+# CONFIG_PROC_KCORE is not set
+CONFIG_PROC_SYSCTL=y
+CONFIG_PROC_PAGE_MONITOR=y
+CONFIG_SYSFS=y
+# CONFIG_TMPFS is not set
+# CONFIG_HUGETLBFS is not set
+# CONFIG_HUGETLB_PAGE is not set
+# CONFIG_CONFIGFS_FS is not set
+# CONFIG_MISC_FILESYSTEMS is not set
+
+#
+# Partition Types
+#
+# CONFIG_PARTITION_ADVANCED is not set
+CONFIG_MSDOS_PARTITION=y
+# CONFIG_NLS is not set
+
+#
+# Kernel hacking
+#
+CONFIG_TRACE_IRQFLAGS_SUPPORT=y
+# CONFIG_PRINTK_TIME is not set
+# CONFIG_ENABLE_WARN_DEPRECATED is not set
+# CONFIG_ENABLE_MUST_CHECK is not set
+CONFIG_FRAME_WARN=2048
+# CONFIG_MAGIC_SYSRQ is not set
+# CONFIG_UNUSED_SYMBOLS is not set
+# CONFIG_DEBUG_FS is not set
+# CONFIG_HEADERS_CHECK is not set
+# CONFIG_DEBUG_KERNEL is not set
+# CONFIG_SLUB_DEBUG_ON is not set
+# CONFIG_SLUB_STATS is not set
+CONFIG_DEBUG_BUGVERBOSE=y
+CONFIG_DEBUG_MEMORY_INIT=y
+CONFIG_ARCH_WANT_FRAME_POINTERS=y
+# CONFIG_FRAME_POINTER is not set
+# CONFIG_RCU_CPU_STALL_DETECTOR is not set
+# CONFIG_LATENCYTOP is not set
+# CONFIG_SYSCTL_SYSCALL_CHECK is not set
+CONFIG_USER_STACKTRACE_SUPPORT=y
+CONFIG_HAVE_FUNCTION_TRACER=y
+CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
+CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST=y
+CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
+CONFIG_HAVE_DYNAMIC_FTRACE=y
+CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
+CONFIG_HAVE_FTRACE_SYSCALLS=y
+CONFIG_TRACING_SUPPORT=y
+# CONFIG_FTRACE is not set
+# CONFIG_DMA_API_DEBUG is not set
+# CONFIG_SAMPLES is not set
+CONFIG_HAVE_ARCH_KGDB=y
+CONFIG_HAVE_ARCH_KMEMCHECK=y
+# CONFIG_STRICT_DEVMEM is not set
+# CONFIG_X86_VERBOSE_BOOTUP is not set
+CONFIG_EARLY_PRINTK=y
+# CONFIG_IOMMU_STRESS is not set
+CONFIG_HAVE_MMIOTRACE_SUPPORT=y
+CONFIG_IO_DELAY_TYPE_0X80=0
+CONFIG_IO_DELAY_TYPE_0XED=1
+CONFIG_IO_DELAY_TYPE_UDELAY=2
+CONFIG_IO_DELAY_TYPE_NONE=3
+CONFIG_IO_DELAY_0X80=y
+# CONFIG_IO_DELAY_0XED is not set
+# CONFIG_IO_DELAY_UDELAY is not set
+# CONFIG_IO_DELAY_NONE is not set
+CONFIG_DEFAULT_IO_DELAY_TYPE=0
+# CONFIG_OPTIMIZE_INLINING is not set
+
+#
+# Security options
+#
+# CONFIG_KEYS is not set
+# CONFIG_SECURITY is not set
+# CONFIG_SECURITYFS is not set
+# CONFIG_SECURITY_FILE_CAPABILITIES is not set
+# CONFIG_CRYPTO is not set
+CONFIG_HAVE_KVM=y
+CONFIG_HAVE_KVM_IRQCHIP=y
+# CONFIG_VIRTUALIZATION is not set
+# CONFIG_BINARY_PRINTF is not set
+
+#
+# Library routines
+#
+CONFIG_GENERIC_FIND_FIRST_BIT=y
+CONFIG_GENERIC_FIND_NEXT_BIT=y
+CONFIG_GENERIC_FIND_LAST_BIT=y
+# CONFIG_CRC_CCITT is not set
+# CONFIG_CRC16 is not set
+# CONFIG_CRC_T10DIF is not set
+# CONFIG_CRC_ITU_T is not set
+# CONFIG_CRC32 is not set
+# CONFIG_CRC7 is not set
+# CONFIG_LIBCRC32C is not set
+CONFIG_HAS_IOMEM=y
+CONFIG_HAS_IOPORT=y
+CONFIG_HAS_DMA=y
.config mini.config differ: byte 1, line 1
Insanity test failed: reversing blank line removal heuristic.
Calculating mini.config...
[37] 0/0 lines 0 bytes 100%
------------------


but my mini.config is still zero bytes.

anybody got a set of steps to follow for miniconfig.sh




regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310307660.0

From rob at landley.net  Mon Jul 11 11:36:03 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 11 Jul 2011 06:36:03 -0500
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <4E194E9B.6010209@sky.com>
References: <4E194E9B.6010209@sky.com>
Message-ID: <4E1AE023.6010402@landley.net>

On 07/10/2011 02:02 AM, Matthew J Fletcher wrote:
> Hi,
> 
> I get this printed several thousand times during compile, it does not
> seem to effect the build process, i.e for various targets it runs to
> completion.

Um, during compile of what?  While building a system image on a host?
Or while building another source package natively under a system image?

> How would i go about finding what sh is trying to invoke ?

I have infrastructure that wraps all the command line utilities and
saves a log of every command line called, in order.  It's described here:

  http://landley.net/aboriginal/FAQ.html#debug_logging

> I am running Mandriva 2010.2 on x86_64

We have an "airlock step" that isolates the build from the host
environment, which is apparently failing here.

In theory the host-tools.sh stage is supposed to build command line
utilities to replace the existing $PATH (mostly busybox), and then point
the $PATH to that new directory (build/host) and nothing else.

That way, the ./configure stages of later packages won't find gettext
and such in the $PATH, thus won't try to use them.  This means that it
won't go "hey, you've got python installed, let me add python support to
the result that will break if python isn't installed on the target".  It
also makes the build more consistent: it does the same thing on fedora
and ubuntu and such because it's building with the same tools (the
busybox ones).  The only things it uses from the host are the compiler
and #!/bin/bash.

This does not seem to be working for you, and I'm not sure why.  Are you
using build.sh, or did you run the build stages individually?

Rob

P.S.  I note that for the Linux From Scratch native build, I'm using a
package to stub out the gettext library support:

URL=ftp://penma.de/code/gettext-stub/gettext-stub-1.tar.gz \
SHA1=ef706667010893c5492173c543d2c5b715abb8a7 \
download || dienow

But that's not for the gettext command line, that's for "man 3 gettext"
(not "man 1 gettext" which you're having trouble with here).

 1310384163.0

From rob at landley.net  Mon Jul 11 13:51:49 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 11 Jul 2011 08:51:49 -0500
Subject: [Aboriginal] Using filesystem images on real hardware
In-Reply-To: <4E195183.8040907@sky.com>
References: <4E195183.8040907@sky.com>
Message-ID: <4E1AFFF5.4080305@landley.net>

On 07/10/2011 02:15 AM, Matthew J Fletcher wrote:
> Hi,
> 
> I've built various filesystem images (armv6l,i486) and run them in qemu
> using the aboriginal build machinery just fine.

Yay!

> Presuming a given specific target board has linux port, e.g Chumby or
> PandaBoard, i guess i just have to create a new target (settings file),

All Aboriginal Linux needs is a new settings file, yes.

What your _target_ is going to need is:

1) binutils+gcc support.

2) libc support (either in uClibc, or you'll need a sources/sections
script to build glibc and changes to simple-cross-compiler.sh and
native-compiler.sh to "build_section glibc" instead of "build_section
uClibc".  Note that glibc requires perl as a build prerequisite, which
is why I added HOST_EXTRA, which is described in the top level "config"
file.)

3) linux support.

If your target is already supported by those four software packages,
you're good.  If it isn't, and you need to supply alternate versions of
binutils/gcc/uClibc/linux.  (There's
USE_UNSTABLE=uClibc,gcc-core,binutils infrastructure for this, but I
really need to update the documentation.  Lemme know if you need to do
that and I'll add a FAQ entry...)

> make sure the gcc parameters are correct for soft/hard float and get a
> kernel config file for settings like the following;

Kernel config and uClibc config.  Plus you need to set KARCH to what the
kernel needs ARCH= set to, and so on.

> CONFIG_CPU_V6=y
> CONFIG_MMU=y
> CONFIG_ARM_THUMB=y
> CONFIG_AEABI=y
> CONFIG_VFP=y

I recently started to automate this, but it's still a bit ugly:

1) Put together your .config file like you want it.
2) run miniconfig.sh on it as described in the FAQ:
  http://landley.net/aboriginal/FAQ.html#dev_miniconfig
3) Use sources/toys/filter.py with the appropriate baseconfig as your
first argument and your new miniconfig as the second.  It'll spit out
all the symbols in the second which weren't in the first.

Then go through it and figure out what you actually _need_...

> # Versatile board
> 
> CONFIG_ARCH_VERSATILE_PB=y
> CONFIG_PCI_LEGACY=y
> CONFIG_SERIAL_NONSTANDARD=y
> CONFIG_SERIAL_AMBA_PL011=y
> CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
> CONFIG_RTC_DRV_PL031=y
> CONFIG_SCSI_SYM53C8XX_2=y
> CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=0
> CONFIG_SCSI_SYM53C8XX_MMIO=y

Assuming your new thing is based on a versatile board, sure.

> how did you find that just these config options need to be set, looking
> at most linux kernel config files there are about 4000 lines in it.

The FAQ explains miniconfig.  Since even most of the miniconfig options
were common between boards I split out the common options into
"baseconfig" and the CONFIG_LINUX stuff gets appended to that, then fed
into the kernel.

> I guess most of those are defaults, do you just diff a default config file
> against a board specific one and see whats different ? then make semi
> intelligent guesses about if those changes are actually target specific
> or just the personal preference of the guy who created the kernel config
> for that board ?

I work out the config I want, which is generally pretty minimal.  Start
with allnoconfig and switch on what you need.  Record what you switched
on.  That's miniconfig.  (The miniconfig.sh script just boils down an
existing config into a miniconfig by removing everything that's at the
allnoconfig default value or which the dependencies would set for you if
you switch on the other symbols.  I need to patch the kconfig C code to
do this for me, the way I'm doing it now is really slow and works it out
experimentally by removing each line and seeing which ones change there
result and which ones don't.)

Having done a bunch of boards, there's some commmon infrastructure like
ext3 that they're all going to need, that's pretty much baseconfig.  I
got that by looking at the dozen or so miniconfigs I had at the time and
factoring out the common symbols.  It's really "what a kernel needs to
run an aboriginal linux system image under qemu".  A real hardware board
running jffs2 or something would need totally different symbols.

Rob

 1310392309.0

From rob at landley.net  Mon Jul 11 14:05:42 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 11 Jul 2011 09:05:42 -0500
Subject: [Aboriginal] kernel build problem for m68k
In-Reply-To: <4E195819.6070307@sky.com>
References: <4E195819.6070307@sky.com>
Message-ID: <4E1B0336.2030508@landley.net>

On 07/10/2011 02:43 AM, Matthew J Fletcher wrote:
> Hi,
> 
> Using aboriginal 1.0.2
> 
> ----
> In file included from arch/m68k/kernel/dma.c:4:
> arch/m68k/kernel/dma_no.c: In function 'dma_sync_single_for_cpu':
> arch/m68k/kernel/dma_no.c:44: error: implicit declaration of function
> 'flush_dcache_range'
> ----

That's either a 2.6.39 bug or my m68k config needs to change for 2.6.39.

> I dont often look in the linux kernel arch stuff, but here goes..
> 
> flush_dcache_range() is a #define in cacheflush_no.h, so it looks like
> <asm/cacheflush.h> is not including cacheflush_no.h
> 
> looking at arch/m68k/include/asm/cacheflush.h there is a big fat #ifdef
> __uClinux__ i guess thats not being fulfilled ?

That should never have gone upstream into Linus's tree, uClinux is a
distro that focuses on nommu system development.  (M68000 processors are
obsolete, so most of the action these days is around a derivative called
"coldfire" that started life as a nommu system and is still mostly used
that way.)

So yeah, kernel bug.  But we need to fix it.

If you can fix it to make it build, make a "-p1" style patch (like git
diff generates, if you have to create 'em yourself diff the "linux"
directories between a vanilla version and your modified version).  Put
the patch in the "sources/patches" directory with a "linux-*.patch"
style name, and then re-run the build.  It should pick it up automatically.

The FAQ has a long, drawn out explanation of the package cache if you
care.  It makes building lots of targets faster, doesn't do so much for
building just one target:

  http://landley.net/aboriginal/FAQ.html#debug_source

Rob

 1310393142.0

From rob at landley.net  Mon Jul 11 14:34:51 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 11 Jul 2011 09:34:51 -0500
Subject: [Aboriginal] Using filesystem images on real hardware
In-Reply-To: <4E19B54C.7070901@sky.com>
References: <4E195183.8040907@sky.com> <20110710173414.341e4540@U64-desktop>
	<4E19B54C.7070901@sky.com>
Message-ID: <4E1B0A0B.4000904@landley.net>

On 07/10/2011 09:21 AM, Matthew J Fletcher wrote:
> On 10/07/11 08:34, David Seikel wrote:
>> On Sun, 10 Jul 2011 08:15:15 +0100 Matthew J Fletcher<amimjf at sky.com>
>> wrote:
>>
>>> how did you find that just these config options need to be set,
>>> looking at most linux kernel config files there are about 4000 lines
>>> in it. I guess most of those are defaults, do you just diff a default
>>> config file against a board specific one and see whats different ?
>>> then make semi intelligent guesses about if those changes are
>>> actually target specific or just the personal preference of the guy
>>> who created the kernel config for that board ?
>>
>> That's called a miniconfig.  Have a look at
>> sources/toys/miniconfig.sh for some details about those.
>>
> 
> Ok, i also looked the comments in the script and the docs at
> http://lwn.net/Articles/160497/ but i think i am missing a bit of
> documentation on how this works.

Did the FAQ help?

> For my board i have a patched 2.6.31 by freescale, it comes with a
> .config file

You'll want to either stick a tarball of the 2.6.31 in
"packages/alt-linux-0.tar.bz2" or just populate the package cache by
hand (put the extracted version at "build/packages/alt-linux" and
_don't_ have a tarball in packages so it won't think it's out of date
and try to update it).

Then export USE_UNSTABLE=linux when you do your build.

There are some sources/targets/alt-linux-*.patch patches, mostly
removing the need for "perl" from the build (2.6.25 added it because
peter anvin is crazy).  If you don't want to patch it you can just set
HOST_EXTRA=qemu (either export it as an environment variable or set it
in config).

> So in the LWN article it says 'create a mini.config file', so i 'touch
> mini.config'.
> 
> I then run 'make allnoconfig KCONFIG_ALLCONFIG=mini.config', my .config
> file is now 14k instead of 17k, but mini.config is still zero bytes.

That converts a miniconfig into a full .config the build can use, you
need to go the other way (which is what miniconfig.sh does, alas very
very slowly.)

> Looking at the usage for miniconfig.sh "Usage: miniconfig.sh
> configfile". I do './miniconfig.sh mini.config', which gives me the
> following output
...
> .config mini.config differ: byte 1, line 1
> Insanity test failed: reversing blank line removal heuristic.

In theory, blank lines and comment lines shouldn't affect the output, so
it tries removing them.

Unfortunately, in current Linux kernels it DOES make a difference, which
is crazy but there you go.  (It doesn't matter in the configs I'm using,
but it does in defconfig.  I forget why.  Make a miniconfig and see
which "#CONFIG_BLAH is not set" lines are still there.  That's the ones
where something funky happens if you don't explicitly switch it off,
even though we started with allnoconfig.)

I've meant to look more closely into this for a while, but my todo list
runneth over.  (I've gotta go to work now, actually.)

> Calculating mini.config...
> [37] 0/0 lines 0 bytes 100%
> ------------------
> 
> 
> but my mini.config is still zero bytes.

Because you started with an empty file, it fed it into allnoconfig, and
then boiled it back down to the resulting miniconfig which was your
empty file back.

> anybody got a set of steps to follow for miniconfig.sh

Lemme know if the FAQ is insufficient, and if so what to fix.

Rob

 1310394891.0

From amimjf at sky.com  Mon Jul 11 18:06:16 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Mon, 11 Jul 2011 19:06:16 +0100
Subject: [Aboriginal] kernel build problem for m68k
In-Reply-To: <4E1B0336.2030508@landley.net>
References: <4E195819.6070307@sky.com> <4E1B0336.2030508@landley.net>
Message-ID: <4E1B3B98.8050104@sky.com>

On 11/07/11 15:05, Rob Landley wrote:
> On 07/10/2011 02:43 AM, Matthew J Fletcher wrote:
>> Hi,
>>
>> Using aboriginal 1.0.2
>>
>> ----
>> In file included from arch/m68k/kernel/dma.c:4:
>> arch/m68k/kernel/dma_no.c: In function 'dma_sync_single_for_cpu':
>> arch/m68k/kernel/dma_no.c:44: error: implicit declaration of function
>> 'flush_dcache_range'
>> ----
>
> That's either a 2.6.39 bug or my m68k config needs to change for 2.6.39.
>
>> I dont often look in the linux kernel arch stuff, but here goes..
>>
>> flush_dcache_range() is a #define in cacheflush_no.h, so it looks like
>> <asm/cacheflush.h>  is not including cacheflush_no.h
>>
>> looking at arch/m68k/include/asm/cacheflush.h there is a big fat #ifdef
>> __uClinux__ i guess thats not being fulfilled ?
>
> That should never have gone upstream into Linus's tree, uClinux is a
> distro that focuses on nommu system development.  (M68000 processors are
> obsolete, so most of the action these days is around a derivative called
> "coldfire" that started life as a nommu system and is still mostly used
> that way.)

Yea i realise m68k is obsolete, it was the arch that powered by first 
computer and what i first started doing embedded work on so i have a 
soft spot for it.


> So yeah, kernel bug.  But we need to fix it.
>
> If you can fix it to make it build, make a "-p1" style patch (like git
> diff generates, if you have to create 'em yourself diff the "linux"
> directories between a vanilla version and your modified version).  Put
> the patch in the "sources/patches" directory with a "linux-*.patch"
> style name, and then re-run the build.  It should pick it up automatically.

I fixed a couple of other nommu related build issues, but it looks like 
m68k gas may be behind the times as well,...

arch/m68k/kernel/entry_no.S: Assembler messages:
arch/m68k/kernel/entry_no.S:47: Error: Unknown operator -- statement 
`save_all' ignored
arch/m68k/kernel/entry_no.S:56: Error: Unknown operator -- statement 
`save_all' ignored


> The FAQ has a long, drawn out explanation of the package cache if you
> care.  It makes building lots of targets faster, doesn't do so much for
> building just one target:
>
>    http://landley.net/aboriginal/FAQ.html#debug_source
>

I brute force and ignorance it by using an SSD :-)


regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310407576.0

From amimjf at sky.com  Mon Jul 11 18:34:49 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Mon, 11 Jul 2011 19:34:49 +0100
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <4E1AE023.6010402@landley.net>
References: <4E194E9B.6010209@sky.com> <4E1AE023.6010402@landley.net>
Message-ID: <4E1B4249.1030005@sky.com>

On 11/07/11 12:36, Rob Landley wrote:
> On 07/10/2011 02:02 AM, Matthew J Fletcher wrote:
>> Hi,
>>
>> I get this printed several thousand times during compile, it does not
>> seem to effect the build process, i.e for various targets it runs to
>> completion.
>
> Um, during compile of what?  While building a system image on a host?
> Or while building another source package natively under a system image?

Almost everything. I started following the FAQ on debug_logging and 
quickly noticed that even "more/record-commands.sh" outputed "sh: 
gettext: command not found"

stripping that script right down to,..

#!/bin/bash
source sources/include.sh || exit 1
$CC --version

my output is,..

[mfletcher at purplebox aboriginal-1.0.2]$ more/record-commands.sh
sh: gettext: command not found
gcc (GCC) 4.4.3
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

interesting,.. not using $CC and doing /usr/bin/gcc --version in the 
script does not complain about gettext.

But inside include.sh all CC is just "export_if_blank CC=cc" and my 'cc' 
is just a symlink to /usr/bin/gcc

I think i am lost.



>> How would i go about finding what sh is trying to invoke ?
>
> I have infrastructure that wraps all the command line utilities and
> saves a log of every command line called, in order.  It's described here:
>
>    http://landley.net/aboriginal/FAQ.html#debug_logging
>
>> I am running Mandriva 2010.2 on x86_64
>
> We have an "airlock step" that isolates the build from the host
> environment, which is apparently failing here.
>
> In theory the host-tools.sh stage is supposed to build command line
> utilities to replace the existing $PATH (mostly busybox), and then point
> the $PATH to that new directory (build/host) and nothing else.
>
> That way, the ./configure stages of later packages won't find gettext
> and such in the $PATH, thus won't try to use them.  This means that it
> won't go "hey, you've got python installed, let me add python support to
> the result that will break if python isn't installed on the target".  It
> also makes the build more consistent: it does the same thing on fedora
> and ubuntu and such because it's building with the same tools (the
> busybox ones).  The only things it uses from the host are the compiler
> and #!/bin/bash.
>
> This does not seem to be working for you, and I'm not sure why.  Are you
> using build.sh, or did you run the build stages individually?
>
> Rob
>
> P.S.  I note that for the Linux From Scratch native build, I'm using a
> package to stub out the gettext library support:
>
> URL=ftp://penma.de/code/gettext-stub/gettext-stub-1.tar.gz \
> SHA1=ef706667010893c5492173c543d2c5b715abb8a7 \
> download || dienow
>
> But that's not for the gettext command line, that's for "man 3 gettext"
> (not "man 1 gettext" which you're having trouble with here).
>


regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310409289.0

From amimjf at sky.com  Mon Jul 11 19:46:17 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Mon, 11 Jul 2011 20:46:17 +0100
Subject: [Aboriginal] Using filesystem images on real hardware
In-Reply-To: <4E1AFFF5.4080305@landley.net>
References: <4E195183.8040907@sky.com> <4E1AFFF5.4080305@landley.net>
Message-ID: <4E1B5309.70805@sky.com>

On 11/07/11 14:51, Rob Landley wrote:
> On 07/10/2011 02:15 AM, Matthew J Fletcher wrote:
>> Hi,
>>
>> I've built various filesystem images (armv6l,i486) and run them in qemu
>> using the aboriginal build machinery just fine.
>
> Yay!
>
>> Presuming a given specific target board has linux port, e.g Chumby or
>> PandaBoard, i guess i just have to create a new target (settings file),
>
> All Aboriginal Linux needs is a new settings file, yes.
>
> What your _target_ is going to need is:
>
> 1) binutils+gcc support.
>
> 2) libc support (either in uClibc, or you'll need a sources/sections
> script to build glibc and changes to simple-cross-compiler.sh and
> native-compiler.sh to "build_section glibc" instead of "build_section
> uClibc".  Note that glibc requires perl as a build prerequisite, which
> is why I added HOST_EXTRA, which is described in the top level "config"
> file.)
>
> 3) linux support.
>
> If your target is already supported by those four software packages,
> you're good.  If it isn't, and you need to supply alternate versions of
> binutils/gcc/uClibc/linux.  (There's
> USE_UNSTABLE=uClibc,gcc-core,binutils infrastructure for this, but I
> really need to update the documentation.  Lemme know if you need to do
> that and I'll add a FAQ entry...)
>
>> make sure the gcc parameters are correct for soft/hard float and get a
>> kernel config file for settings like the following;
>
> Kernel config and uClibc config.  Plus you need to set KARCH to what the
> kernel needs ARCH= set to, and so on.
>
>> CONFIG_CPU_V6=y
>> CONFIG_MMU=y
>> CONFIG_ARM_THUMB=y
>> CONFIG_AEABI=y
>> CONFIG_VFP=y
>
> I recently started to automate this, but it's still a bit ugly:
>
> 1) Put together your .config file like you want it.
> 2) run miniconfig.sh on it as described in the FAQ:
>    http://landley.net/aboriginal/FAQ.html#dev_miniconfig
> 3) Use sources/toys/filter.py with the appropriate baseconfig as your
> first argument and your new miniconfig as the second.  It'll spit out
> all the symbols in the second which weren't in the first.
>
> Then go through it and figure out what you actually _need_...
>
>> # Versatile board
>>
>> CONFIG_ARCH_VERSATILE_PB=y
>> CONFIG_PCI_LEGACY=y
>> CONFIG_SERIAL_NONSTANDARD=y
>> CONFIG_SERIAL_AMBA_PL011=y
>> CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
>> CONFIG_RTC_DRV_PL031=y
>> CONFIG_SCSI_SYM53C8XX_2=y
>> CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=0
>> CONFIG_SCSI_SYM53C8XX_MMIO=y
>
> Assuming your new thing is based on a versatile board, sure.
>
>> how did you find that just these config options need to be set, looking
>> at most linux kernel config files there are about 4000 lines in it.
>
> The FAQ explains miniconfig.  Since even most of the miniconfig options
> were common between boards I split out the common options into
> "baseconfig" and the CONFIG_LINUX stuff gets appended to that, then fed
> into the kernel.
>
>> I guess most of those are defaults, do you just diff a default config file
>> against a board specific one and see whats different ? then make semi
>> intelligent guesses about if those changes are actually target specific
>> or just the personal preference of the guy who created the kernel config
>> for that board ?
>
> I work out the config I want, which is generally pretty minimal.  Start
> with allnoconfig and switch on what you need.  Record what you switched
> on.  That's miniconfig.  (The miniconfig.sh script just boils down an
> existing config into a miniconfig by removing everything that's at the
> allnoconfig default value or which the dependencies would set for you if
> you switch on the other symbols.  I need to patch the kconfig C code to
> do this for me, the way I'm doing it now is really slow and works it out
> experimentally by removing each line and seeing which ones change there
> result and which ones don't.)
>
> Having done a bunch of boards, there's some commmon infrastructure like
> ext3 that they're all going to need, that's pretty much baseconfig.  I
> got that by looking at the dozen or so miniconfigs I had at the time and
> factoring out the common symbols.  It's really "what a kernel needs to
> run an aboriginal linux system image under qemu".  A real hardware board
> running jffs2 or something would need totally different symbols.
>
> Rob
>

Rob, thanks for the help.

Ok so i start with the freescale 2.6.31 patched up sources with a 
imx25_3stack_defconfig for the board.

So reading the FAQ i run 'make allnoconfig KCONFIG_ALLCONFIG=filename', 
so i am guessing if filename is an input or an output ? well it does not 
produce a file called 'filename', and the '.config' created is for a 
x86_64 build which is my host.

So i presume 'filename' is an input, so i pass it the filename of my 
defconfig. Humm, that created a '.config' file for x86_64 as well.

Ok, i will replace '.config' with my defconfig and try again. 'cp 
imx25_3stack_defconfig .config', then do 'make allnoconfig 
KCONFIG_ALLCONFIG=filename', humm its re-written my Arm defconfig to be 
x86_64 again.

Not making much progress here. Whatever i do the 'make allnoconfig' 
seems to make a x86_64 '.config' file.



regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310413577.0

From rob at landley.net  Mon Jul 11 23:27:30 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 11 Jul 2011 18:27:30 -0500
Subject: [Aboriginal] Using filesystem images on real hardware
In-Reply-To: <4E1B5309.70805@sky.com>
References: <4E195183.8040907@sky.com> <4E1AFFF5.4080305@landley.net>
	<4E1B5309.70805@sky.com>
Message-ID: <4E1B86E2.3000508@landley.net>

On 07/11/2011 02:46 PM, Matthew J Fletcher wrote:
>> Having done a bunch of boards, there's some commmon infrastructure like
>> ext3 that they're all going to need, that's pretty much baseconfig.  I
>> got that by looking at the dozen or so miniconfigs I had at the time and
>> factoring out the common symbols.  It's really "what a kernel needs to
>> run an aboriginal linux system image under qemu".  A real hardware board
>> running jffs2 or something would need totally different symbols.
>>
>> Rob
>>
> 
> Rob, thanks for the help.
> 
> Ok so i start with the freescale 2.6.31 patched up sources with a
> imx25_3stack_defconfig for the board.

Ok.

> So reading the FAQ i run 'make allnoconfig KCONFIG_ALLCONFIG=filename',

That turns a miniconfig into a full .config.  The "filename" in that is
the miniconfig file.

> so i am guessing if filename is an input or an output ? well it does not
> produce a file called 'filename', and the '.config' created is for a
> x86_64 build which is my host.

You always have to specify ARCH= if you're cross compiling.  (That's
just how building the kernel works.)

> So i presume 'filename' is an input, so i pass it the filename of my
> defconfig. Humm, that created a '.config' file for x86_64 as well.

No, of your miniconfig.  This is what _consumes_ a miniconfig to create
a full config.  The "miniconfig.sh" script goes the other way, it's what
consumes a full config to create a miniconfig.

> Ok, i will replace '.config' with my defconfig and try again. 'cp
> imx25_3stack_defconfig .config', then do 'make allnoconfig
> KCONFIG_ALLCONFIG=filename', humm its re-written my Arm defconfig to be
> x86_64 again.
> 
> Not making much progress here. Whatever i do the 'make allnoconfig'
> seems to make a x86_64 '.config' file.

cd ~yourlinux

ARCH=arm ~/aboriginal/sources/toys/miniconfig.sh ~/yourdefconfig

~/aboriginal/sources/toys/filter.py ~/aboriginal/baseconfig-linux \
  mini.config > newsymbols.txt

Rob

 1310426850.0

From rob at landley.net  Tue Jul 12 00:15:53 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 11 Jul 2011 19:15:53 -0500
Subject: [Aboriginal] kernel build problem for m68k
In-Reply-To: <4E1B3B98.8050104@sky.com>
References: <4E195819.6070307@sky.com> <4E1B0336.2030508@landley.net>
	<4E1B3B98.8050104@sky.com>
Message-ID: <4E1B9239.7030502@landley.net>

On 07/11/2011 01:06 PM, Matthew J Fletcher wrote:
> On 11/07/11 15:05, Rob Landley wrote:
>> On 07/10/2011 02:43 AM, Matthew J Fletcher wrote:
>>> Hi,
>>>
>>> Using aboriginal 1.0.2
>>>
>>> ----
>>> In file included from arch/m68k/kernel/dma.c:4:
>>> arch/m68k/kernel/dma_no.c: In function 'dma_sync_single_for_cpu':
>>> arch/m68k/kernel/dma_no.c:44: error: implicit declaration of function
>>> 'flush_dcache_range'
>>> ----
>>
>> That's either a 2.6.39 bug or my m68k config needs to change for 2.6.39.
>>
>>> I dont often look in the linux kernel arch stuff, but here goes..
>>>
>>> flush_dcache_range() is a #define in cacheflush_no.h, so it looks like
>>> <asm/cacheflush.h>  is not including cacheflush_no.h
>>>
>>> looking at arch/m68k/include/asm/cacheflush.h there is a big fat #ifdef
>>> __uClinux__ i guess thats not being fulfilled ?
>>
>> That should never have gone upstream into Linus's tree, uClinux is a
>> distro that focuses on nommu system development.  (M68000 processors are
>> obsolete, so most of the action these days is around a derivative called
>> "coldfire" that started life as a nommu system and is still mostly used
>> that way.)
> 
> Yea i realise m68k is obsolete, it was the arch that powered by first
> computer and what i first started doing embedded work on so i have a
> soft spot for it.

Understood.  I want to get Alpha working too, I just haven't had time...

>> So yeah, kernel bug.  But we need to fix it.
>>
>> If you can fix it to make it build, make a "-p1" style patch (like git
>> diff generates, if you have to create 'em yourself diff the "linux"
>> directories between a vanilla version and your modified version).  Put
>> the patch in the "sources/patches" directory with a "linux-*.patch"
>> style name, and then re-run the build.  It should pick it up
>> automatically.
> 
> I fixed a couple of other nommu related build issues, but it looks like
> m68k gas may be behind the times as well,...

I'm still using binutils 2.17 (because that's the last GPLv2 release and
I really don't want to open the can of worms that is GPLv3.)

> arch/m68k/kernel/entry_no.S: Assembler messages:
> arch/m68k/kernel/entry_no.S:47: Error: Unknown operator -- statement
> `save_all' ignored
> arch/m68k/kernel/entry_no.S:56: Error: Unknown operator -- statement
> `save_all' ignored

Sigh.  It build under 2.6.37.  I think it did under .38.  I'll have to
bisect and see where this was introduced...

(The m68k target did at one point work under aranym, I just have trouble
regression testing it without qemu support.  My automated tests always
list it as "fail" because it doesn't run...)

>> The FAQ has a long, drawn out explanation of the package cache if you
>> care.  It makes building lots of targets faster, doesn't do so much for
>> building just one target:
>>
>>    http://landley.net/aboriginal/FAQ.html#debug_source
>>
> 
> I brute force and ignorance it by using an SSD :-)

It's actually not just disk speed, on a sequential build the killer is
the extra CPU of re-extracting all those tarballs, plus the extra I/O
and page cache from faulting in lots of different copies of the same
files into memory.  Even if your system has enough memory you can build
out of ramfs (which my friend Mark has), it's a very noticeable
performance difference.  (And when building all targets in parallel with
FORK=1 it's _huge_, due to the different instances of the same cached
data thrashing the L2 cache...)

Rob

 1310429753.0

From rob at landley.net  Tue Jul 12 00:24:12 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 11 Jul 2011 19:24:12 -0500
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <4E1B4249.1030005@sky.com>
References: <4E194E9B.6010209@sky.com> <4E1AE023.6010402@landley.net>
	<4E1B4249.1030005@sky.com>
Message-ID: <4E1B942C.6050004@landley.net>

On 07/11/2011 01:34 PM, Matthew J Fletcher wrote:
> On 11/07/11 12:36, Rob Landley wrote:
>> On 07/10/2011 02:02 AM, Matthew J Fletcher wrote:
>>> Hi,
>>>
>>> I get this printed several thousand times during compile, it does not
>>> seem to effect the build process, i.e for various targets it runs to
>>> completion.
>>
>> Um, during compile of what?  While building a system image on a host?
>> Or while building another source package natively under a system image?
> 
> Almost everything. I started following the FAQ on debug_logging and
> quickly noticed that even "more/record-commands.sh" outputed "sh:
> gettext: command not found"

Huh.  Why does it _think_ you have gettext?  If you ran host-tools.sh
there shouldn't be one in build/host.  (Check your build/host to see if
you have one?)

Do "which gettext" and see what it finds?  And if you can run it?

> stripping that script right down to,..
> 
> #!/bin/bash
> source sources/include.sh || exit 1
> $CC --version
> 
> my output is,..
> 
> [mfletcher at purplebox aboriginal-1.0.2]$ more/record-commands.sh
> sh: gettext: command not found
> gcc (GCC) 4.4.3
> Copyright (C) 2010 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> 
> interesting,.. not using $CC and doing /usr/bin/gcc --version in the
> script does not complain about gettext.

Sigh.  Ubuntu made its horrible wrapper script even worse, didn't it?

Do you suppose it has any connection to this old commit?

  http://landley.net/hg/aboriginal/rev/1041

> But inside include.sh all CC is just "export_if_blank CC=cc" and my 'cc'
> is just a symlink to /usr/bin/gcc
> 
> I think i am lost.

Ubuntu turned gcc into a perl script that calls gcc.  (Yes really.)
Sounds like Mandrake might have done the same thing, if so I'll have to
install a copy here and fix it...

Now you see why I go to such great lengths to sanitize the host
environment.  Because they keep coming up with new ways to be CRAZY.  I
can work around everything but the host toolchain by building busybox (I
spent several years implementing working versions of things like "sort"
and "sed" and "patch" and "mount" in busybox), but unfortunately you
need a compiler to build a compiler...

>>> How would i go about finding what sh is trying to invoke ?
>>
>> I have infrastructure that wraps all the command line utilities and
>> saves a log of every command line called, in order.  It's described here:
>>
>>    http://landley.net/aboriginal/FAQ.html#debug_logging

Note that shell scripts which call #!/stuff bypass the $PATH and thus my
wrapper won't catch those.

Rob

 1310430252.0

From rob at landley.net  Tue Jul 12 00:28:56 2011
From: rob at landley.net (Rob Landley)
Date: Mon, 11 Jul 2011 19:28:56 -0500
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <4E194E9B.6010209@sky.com>
References: <4E194E9B.6010209@sky.com>
Message-ID: <4E1B9548.5090902@landley.net>

On 07/10/2011 02:02 AM, Matthew J Fletcher wrote:

> I am running Mandriva 2010.2 on x86_64

Not the world's easiest distro to download.  I navigated past the "pay
version, subscribers only" bit to find the free versino, which went:

http://ftp.ndlug.nd.edu/pub/Mandrake/official/iso/2010.2/mandriva-linux-free-2010.2-x86_64.iso

Forbidden

You don't have permission to access
/pub/Mandrake/official/iso/2010.2/mandriva-linux-free-2010.2-x86_64.iso
on this server.

Additionally, a 403 Forbidden error was encountered while trying to use
an ErrorDocument to handle the request.

Rob

 1310430536.0

From bjorn.forsman at gmail.com  Tue Jul 12 18:31:48 2011
From: bjorn.forsman at gmail.com (=?UTF-8?Q?Bj=C3=B8rn_Forsman?=)
Date: Tue, 12 Jul 2011 20:31:48 +0200
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <4E1B942C.6050004@landley.net>
References: <4E194E9B.6010209@sky.com> <4E1AE023.6010402@landley.net>
	<4E1B4249.1030005@sky.com> <4E1B942C.6050004@landley.net>
Message-ID: <CAEYzJUF-eqSFLcmoZn=JRHhQe8nJXOff=Z3tDjaznLxcebwzyg@mail.gmail.com>

On 12 July 2011 02:24, Rob Landley <rob at landley.net> wrote:
> Ubuntu turned gcc into a perl script that calls gcc. ?(Yes really.)

FYI, gcc isn't a perl script in Ubuntu 11.04:

$ which gcc
/usr/bin/gcc
$ file /usr/bin/gcc
/usr/bin/gcc: symbolic link to `gcc-4.5'
$ file /usr/bin/gcc-4.5
/usr/bin/gcc-4.5: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
dynamically linked (uses shared libs), for GNU/Linux 2.6.15, stripped

Best regards,
Bj?rn Forsman

 1310495508.0

From amimjf at sky.com  Tue Jul 12 20:02:07 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Tue, 12 Jul 2011 21:02:07 +0100
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <4E1B942C.6050004@landley.net>
References: <4E194E9B.6010209@sky.com> <4E1AE023.6010402@landley.net>
	<4E1B4249.1030005@sky.com> <4E1B942C.6050004@landley.net>
Message-ID: <4E1CA83F.3040700@sky.com>

On 12/07/11 01:24, Rob Landley wrote:
> On 07/11/2011 01:34 PM, Matthew J Fletcher wrote:
>> On 11/07/11 12:36, Rob Landley wrote:
>>> On 07/10/2011 02:02 AM, Matthew J Fletcher wrote:
>>>> Hi,
>>>>
>>>> I get this printed several thousand times during compile, it does not
>>>> seem to effect the build process, i.e for various targets it runs to
>>>> completion.
>>>
>>> Um, during compile of what?  While building a system image on a host?
>>> Or while building another source package natively under a system image?
>>
>> Almost everything. I started following the FAQ on debug_logging and
>> quickly noticed that even "more/record-commands.sh" outputed "sh:
>> gettext: command not found"
>
> Huh.  Why does it _think_ you have gettext?  If you ran host-tools.sh
> there shouldn't be one in build/host.  (Check your build/host to see if
> you have one?)
>
> Do "which gettext" and see what it finds?  And if you can run it?

[mfletcher at purplebox ~]$ which gettext
/usr/bin/gettext

[mfletcher at purplebox ~]$ gettext --version
gettext (GNU gettext-runtime) 0.17
Copyright (C) 1995-1997, 2000-2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

but i also have,.

[mfletcher at purplebox ~]$ which gettext.sh
/usr/bin/gettext.sh

[mfletcher at purplebox ~]$ gettext.sh --version
/usr/bin/gettext.sh (GNU gettext-runtime) 0.17
Copyright (C) 2003-2007 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later 
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

bonkers !


>> stripping that script right down to,..
>>
>> #!/bin/bash
>> source sources/include.sh || exit 1
>> $CC --version
>>
>> my output is,..
>>
>> [mfletcher at purplebox aboriginal-1.0.2]$ more/record-commands.sh
>> sh: gettext: command not found
>> gcc (GCC) 4.4.3
>> Copyright (C) 2010 Free Software Foundation, Inc.
>> This is free software; see the source for copying conditions.  There is NO
>> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>>
>> interesting,.. not using $CC and doing /usr/bin/gcc --version in the
>> script does not complain about gettext.
>
> Sigh.  Ubuntu made its horrible wrapper script even worse, didn't it?
>
> Do you suppose it has any connection to this old commit?
>
>    http://landley.net/hg/aboriginal/rev/1041
>
>> But inside include.sh all CC is just "export_if_blank CC=cc" and my 'cc'
>> is just a symlink to /usr/bin/gcc
>>
>> I think i am lost.
>
> Ubuntu turned gcc into a perl script that calls gcc.  (Yes really.)
> Sounds like Mandrake might have done the same thing, if so I'll have to
> install a copy here and fix it...

On this box /usr/bin/gcc which simlinks to cc is a binary, nano says the 
first bytes are ^?ELF.

you probably wont like my linux box at work that uses the shell wrapper 
colourgcc (or it might be the american spelling), anyhow thats a 
different box.


> Now you see why I go to such great lengths to sanitize the host
> environment.  Because they keep coming up with new ways to be CRAZY.  I
> can work around everything but the host toolchain by building busybox (I
> spent several years implementing working versions of things like "sort"
> and "sed" and "patch" and "mount" in busybox), but unfortunately you
> need a compiler to build a compiler...
>
>>>> How would i go about finding what sh is trying to invoke ?
>>>
>>> I have infrastructure that wraps all the command line utilities and
>>> saves a log of every command line called, in order.  It's described here:
>>>
>>>     http://landley.net/aboriginal/FAQ.html#debug_logging
>
> Note that shell scripts which call #!/stuff bypass the $PATH and thus my
> wrapper won't catch those.
>
> Rob
>


regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310500927.0

From amimjf at sky.com  Tue Jul 12 21:04:40 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Tue, 12 Jul 2011 22:04:40 +0100
Subject: [Aboriginal] Using filesystem images on real hardware
In-Reply-To: <4E1B86E2.3000508@landley.net>
References: <4E195183.8040907@sky.com> <4E1AFFF5.4080305@landley.net>
	<4E1B5309.70805@sky.com> <4E1B86E2.3000508@landley.net>
Message-ID: <4E1CB6E8.7000407@sky.com>

On 12/07/11 00:27, Rob Landley wrote:
> On 07/11/2011 02:46 PM, Matthew J Fletcher wrote:
>>> Having done a bunch of boards, there's some commmon infrastructure like
>>> ext3 that they're all going to need, that's pretty much baseconfig.  I
>>> got that by looking at the dozen or so miniconfigs I had at the time and
>>> factoring out the common symbols.  It's really "what a kernel needs to
>>> run an aboriginal linux system image under qemu".  A real hardware board
>>> running jffs2 or something would need totally different symbols.
>>>
>>> Rob
>>>
>>
>> Rob, thanks for the help.
>>
>> Ok so i start with the freescale 2.6.31 patched up sources with a
>> imx25_3stack_defconfig for the board.
>
> Ok.
>
>> So reading the FAQ i run 'make allnoconfig KCONFIG_ALLCONFIG=filename',
>
> That turns a miniconfig into a full .config.  The "filename" in that is
> the miniconfig file.
>
>> so i am guessing if filename is an input or an output ? well it does not
>> produce a file called 'filename', and the '.config' created is for a
>> x86_64 build which is my host.
>
> You always have to specify ARCH= if you're cross compiling.  (That's
> just how building the kernel works.)
>
>> So i presume 'filename' is an input, so i pass it the filename of my
>> defconfig. Humm, that created a '.config' file for x86_64 as well.
>
> No, of your miniconfig.  This is what _consumes_ a miniconfig to create
> a full config.  The "miniconfig.sh" script goes the other way, it's what
> consumes a full config to create a miniconfig.
>
>> Ok, i will replace '.config' with my defconfig and try again. 'cp
>> imx25_3stack_defconfig .config', then do 'make allnoconfig
>> KCONFIG_ALLCONFIG=filename', humm its re-written my Arm defconfig to be
>> x86_64 again.
>>
>> Not making much progress here. Whatever i do the 'make allnoconfig'
>> seems to make a x86_64 '.config' file.
>
> cd ~yourlinux
>
> ARCH=arm ~/aboriginal/sources/toys/miniconfig.sh ~/yourdefconfig
>
> ~/aboriginal/sources/toys/filter.py ~/aboriginal/baseconfig-linux \
>    mini.config>  newsymbols.txt
>
> Rob
>

Getting there, "> ARCH=arm ~/aboriginal/sources/toys/miniconfig.sh 
~/yourdefconfig" worked, produced a nice cut down mini.config file

'filter.py' was not in the 1.0.2 tarball but i grabbed it from online 
source repository.

filter.py made no changes to the mini.conf, but it was pretty small anyway.

my settings file for my new target is a combination of armv6l with the 
config settings from the mini.conf


Rob, thanks for all the help, whilst waiting for the build i was reading 
your blog about how the English (me apparently !) make Tea, well i make 
lots of tea, but i don't like drinking it myself so have pretty much no 
idea what it tastes like !

Humm a couple of compile errors right after creating the squashfs 
filesystem.

/home/scx/aboriginal-1.0.2/sources/targets/3stack/settings: line 90: rw: 
command not found
=== linux (3stack linux-kernel)

there is a pretty odd looking config line,

CONFIG_CMDLINE="noinitrd console=ttymxc0,115200 root=/dev/mtdblock2 rw 
rootfstype=jffs2 ip=off"

is some tool not parsing that 'rw' is an option and not a command ?

with that config option chopped out it builds further,. with a slightly 
ominous warning,.

arch/arm/kernel/unwind.c:35:2: warning: #warning Your compiler is too 
buggy; it is known to not compile ARM unwind support.
arch/arm/kernel/unwind.c:36:2: warning: #warning Change compiler or 
disable ARM_UNWIND option.

I realise the GPL3, newer GCC's/binutils etc is a slightly delicate subject.


regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310504680.0

From rob at landley.net  Wed Jul 13 01:17:39 2011
From: rob at landley.net (Rob Landley)
Date: Tue, 12 Jul 2011 20:17:39 -0500
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <CAEYzJUF-eqSFLcmoZn=JRHhQe8nJXOff=Z3tDjaznLxcebwzyg@mail.gmail.com>
References: <4E194E9B.6010209@sky.com>	<4E1AE023.6010402@landley.net>	<4E1B4249.1030005@sky.com>	<4E1B942C.6050004@landley.net>
	<CAEYzJUF-eqSFLcmoZn=JRHhQe8nJXOff=Z3tDjaznLxcebwzyg@mail.gmail.com>
Message-ID: <4E1CF233.1000609@landley.net>

On 07/12/2011 01:31 PM, Bj?rn Forsman wrote:
> On 12 July 2011 02:24, Rob Landley <rob at landley.net> wrote:
>> Ubuntu turned gcc into a perl script that calls gcc.  (Yes really.)
> 
> FYI, gcc isn't a perl script in Ubuntu 11.04:
> 
> $ which gcc
> /usr/bin/gcc
> $ file /usr/bin/gcc
> /usr/bin/gcc: symbolic link to `gcc-4.5'
> $ file /usr/bin/gcc-4.5
> /usr/bin/gcc-4.5: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
> dynamically linked (uses shared libs), for GNU/Linux 2.6.15, stripped
> 
> Best regards,
> Bj?rn Forsman

Good to know they reverted that.  It was crazy.  (Or possibly it doesn't
kick in unless you install distcc or ccache or some such...)

Rob

 1310519859.0

From rob at landley.net  Wed Jul 13 12:48:10 2011
From: rob at landley.net (Rob Landley)
Date: Wed, 13 Jul 2011 07:48:10 -0500
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <4E1CA83F.3040700@sky.com>
References: <4E194E9B.6010209@sky.com>
	<4E1AE023.6010402@landley.net>	<4E1B4249.1030005@sky.com>
	<4E1B942C.6050004@landley.net> <4E1CA83F.3040700@sky.com>
Message-ID: <4E1D940A.3020504@landley.net>

On 07/12/2011 03:02 PM, Matthew J Fletcher wrote:
>> Ubuntu turned gcc into a perl script that calls gcc.  (Yes really.)
>> Sounds like Mandrake might have done the same thing, if so I'll have to
>> install a copy here and fix it...
> 
> On this box /usr/bin/gcc which simlinks to cc is a binary, nano says the
> first bytes are ^?ELF.

So that's not it...

I note that leaving mandrivia running overnight trying to format the
virtual hard drive didn't complete.  Sigh...

> you probably wont like my linux box at work that uses the shell wrapper
> colourgcc (or it might be the american spelling), anyhow thats a
> different box.

As long as it doesn't call anything that's not in busybox, it should
work fine...

The point of isolating the $PATH is to prevent ./configure scripts from
finding stuff out of the host that won't be on the target.
Internationalization support is one of them, and that's what gettext is
part of.

It seems like it's just a cosmetic problem, but I still want to fix it.
 If I can ever manage to reproduce it...

Rob

 1310561290.0

From amimjf at sky.com  Wed Jul 13 19:12:49 2011
From: amimjf at sky.com (Matthew J Fletcher)
Date: Wed, 13 Jul 2011 20:12:49 +0100
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <4E1D940A.3020504@landley.net>
References: <4E194E9B.6010209@sky.com>
	<4E1AE023.6010402@landley.net>	<4E1B4249.1030005@sky.com>
	<4E1B942C.6050004@landley.net> <4E1CA83F.3040700@sky.com>
	<4E1D940A.3020504@landley.net>
Message-ID: <4E1DEE31.1090803@sky.com>

On 13/07/11 13:48, Rob Landley wrote:
> On 07/12/2011 03:02 PM, Matthew J Fletcher wrote:
>>> Ubuntu turned gcc into a perl script that calls gcc.  (Yes really.)
>>> Sounds like Mandrake might have done the same thing, if so I'll have to
>>> install a copy here and fix it...
>>
>> On this box /usr/bin/gcc which simlinks to cc is a binary, nano says the
>> first bytes are ^?ELF.
>
> So that's not it...
>
> I note that leaving mandrivia running overnight trying to format the
> virtual hard drive didn't complete.  Sigh...

Used Virtualbox for years, works find with pretty much all the distros 
and Windows versions i have tried. Mandriva included.


>> you probably wont like my linux box at work that uses the shell wrapper
>> colourgcc (or it might be the american spelling), anyhow thats a
>> different box.
>
> As long as it doesn't call anything that's not in busybox, it should
> work fine...
>
> The point of isolating the $PATH is to prevent ./configure scripts from
> finding stuff out of the host that won't be on the target.
> Internationalization support is one of them, and that's what gettext is
> part of.
>
> It seems like it's just a cosmetic problem, but I still want to fix it.
>   If I can ever manage to reproduce it...
>
> Rob



regards

---
Matthew J Fletcher
amimjf(at)sky.com
www.amimjf.org
---

 1310584369.0

From earlenceferns at gmail.com  Fri Jul 15 20:30:26 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Fri, 15 Jul 2011 22:30:26 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C compiler
Message-ID: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>

Hello,

I plan on using aboriginal linux scripts to build a C compiler thats
customized for Android.
Therefore, I think these steps would be needed:

1. download.sh to get GCC sources.
2. patch them with android specific headers and implementation
3. native-build.sh for ARM

Next, I need to optimized for space, I need it as small as possible.
Therefore, disable CPP (with the given option),
remove pthreads, binutils etc etc -> but how do I config this?
I basically need to be really lean and mean. C compiler, libc, libm, linux
headers. thats it.

What do you think?

Cheers,
Earlence
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110715/f24199d8/attachment.htm>

From earlenceferns at gmail.com  Sat Jul 16 12:53:59 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Sat, 16 Jul 2011 14:53:59 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
Message-ID: <CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>

Ok, I;ve built from source, the native compiler for armv5l. It compiles a
couple of files I need on the devices perfectly.
However, One particular file, I use netlink sockets and this is the message
I get:

/data/local/tmp/narm/bin/../include/linux/netlink.h:31: error: expected
specifier-qualifier-list before 'sa_family_t'

gb.c:310: error: 'struct sockaddr_nl' has no member named 'nl_family'
gb.c:314: error: 'struct sockaddr_nl' has no member named 'nl_pid'

(note, this is executed on a Samsung Galaxy S running Gingerbread)

-Earlence

On Fri, Jul 15, 2011 at 10:30 PM, Earlence Fernandes <
earlenceferns at gmail.com> wrote:

> Hello,
>
> I plan on using aboriginal linux scripts to build a C compiler thats
> customized for Android.
> Therefore, I think these steps would be needed:
>
> 1. download.sh to get GCC sources.
> 2. patch them with android specific headers and implementation
> 3. native-build.sh for ARM
>
> Next, I need to optimized for space, I need it as small as possible.
> Therefore, disable CPP (with the given option),
> remove pthreads, binutils etc etc -> but how do I config this?
> I basically need to be really lean and mean. C compiler, libc, libm, linux
> headers. thats it.
>
> What do you think?
>
> Cheers,
> Earlence
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110716/0e5f2e39/attachment.htm>

From rob at landley.net  Sat Jul 16 16:08:51 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 16 Jul 2011 11:08:51 -0500
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
Message-ID: <4E21B793.1030501@landley.net>

On 07/15/2011 03:30 PM, Earlence Fernandes wrote:
> Hello,
> 
> I plan on using aboriginal linux scripts to build a C compiler thats
> customized for Android.

Android is generally armv5l or similar, it just uses a different C
library (bionic instead of uClibc).  I note that uClibc works just fine
on android (if you either statically link or install the shared
libraries), but if you want to build everything with bionic that's fine too.

The question for a self-hosting system is do the other six packages
build with bionic?  (gcc, binutils, busybox, make, bash.  The linux
kernel won't care.)

If you're not doing a self-hosting system, it shouldn't matter much what
libc you use as long as the compiler can find it.

I gave up trying to fix the path logic in gcc years ago, instead I use a
wrapper (sources/toys/ccwrap.c) which parses the incoming gcc command
line and rewrites it to start with --nostdinc --nostdlib and then adds
in the explicit header and library paths it should search.  When it's
not a -c, -E, or -S build it explicitly adds in the libraries and .o
files necessary for linking executables.

That expects the compiler binary to be in "bin" and looks for ../include
and ../cc/include for headers, and ../lib and ../cc/lib for libraries.
In the case of linking in libc, it looks for the name "libc.so" which
should be symlinked to the C library you want to use.

> Therefore, I think these steps would be needed:
> 
> 1. download.sh to get GCC sources.

You can just populate the packages directory yourself and skip
download.sh if you like.  But sure, modifying that would work.

> 2. patch them with android specific headers and implementation

I note that sources/patches exists to apply patches to upstream
tarballs.  The extract logic will automatically apply patches with the
right names ("packagename-*.patch", the required dash is so uClibc
doesn't get uClibc++ patches applied to it).

> 3. native-build.sh for ARM

In theory that code is completely target-agnostic.  (And the actual
individual package builds are in sources/sections so they can be re-used
by the various stages as necessary.)

In practice it's not necessarily _version_ agnostic: gcc 4.2 was just
two packages, gcc 4.3 got split into four packages, and gcc 4.5 is
written in C++ so requires a different host compiler setup.

(The people who wrote gcc are completely insane, I'm really looking
forward to llvm or pcc or something becoming usable to build Linux
systems with.)

> Next, I need to optimized for space, I need it as small as possible.
> Therefore, disable CPP (with the given option),

Actually cpp is the C preprocessor (I.E. what gcc -E calls out to), you
want to disable c++.

> remove pthreads, binutils etc etc -> but how do I config this?

You can't remove binutils, it's the linker and the assembler.  (No, I
don't know why the gcc developers split the linker and assembler out
into a separate package.  I don't know why the split out GMP and MPFR in
4.3 either, but since it's GPLv3 I'm not comfortable shipping binaries
built from it so it hasn't come up.  I note that Linux From Scratch has
instructions for building those...)

> I basically need to be really lean and mean. C compiler, libc, libm,
> linux headers. thats it.

Have you looked at tinycc?  It's a working C compiler (including linker
and assembler all in one binary, busybox-style) in 100k.  Last I checked
they claimed to have basic arm support working.  (Arm on an arm host
didn't work for me, but that was 3 years ago...)

Rob

 1310832531.0

From earlenceferns at gmail.com  Sat Jul 16 16:15:42 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Sat, 16 Jul 2011 18:15:42 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <4E21B793.1030501@landley.net>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<4E21B793.1030501@landley.net>
Message-ID: <CA+ve00R0-cFZcBVZZJg92_bfPkXH3xFccNNva+D78Kf_FMywgw@mail.gmail.com>

>Have you looked at tinycc?
Yes, but theres not much development going on there and theres a lack of
support on the mailing lists.

As I said, I managed to build it around 35MB (is fine for me) and the basic
"hello world" works on the device.
I also patched in code for android that I need (like system_properties.c)
(and it works)

The only error I'm getting now is ->

/data/local/tmp/narm/bin/../include/linux/netlink.h:40: error: expected
specifier-qualifier-list before 'sa_family_t'
gb.c: In function 'last_try':
gb.c:310: error: 'struct sockaddr_nl' has no member named 'nl_family'
gb.c:314: error: 'struct sockaddr_nl' has no member named 'nl_pid'
gb.c: In function 'do_fault':
gb.c:347: error: 'struct sockaddr_nl' has no member named 'nl_family'
gb.c:351: error: 'struct sockaddr_nl' has no member named 'nl_pid'


I suspect this is not android specific. Can you help me out here?

-Earlence



On Sat, Jul 16, 2011 at 6:08 PM, Rob Landley <rob at landley.net> wrote:

> On 07/15/2011 03:30 PM, Earlence Fernandes wrote:
> > Hello,
> >
> > I plan on using aboriginal linux scripts to build a C compiler thats
> > customized for Android.
>
> Android is generally armv5l or similar, it just uses a different C
> library (bionic instead of uClibc).  I note that uClibc works just fine
> on android (if you either statically link or install the shared
> libraries), but if you want to build everything with bionic that's fine
> too.
>
> The question for a self-hosting system is do the other six packages
> build with bionic?  (gcc, binutils, busybox, make, bash.  The linux
> kernel won't care.)
>
> If you're not doing a self-hosting system, it shouldn't matter much what
> libc you use as long as the compiler can find it.
>
> I gave up trying to fix the path logic in gcc years ago, instead I use a
> wrapper (sources/toys/ccwrap.c) which parses the incoming gcc command
> line and rewrites it to start with --nostdinc --nostdlib and then adds
> in the explicit header and library paths it should search.  When it's
> not a -c, -E, or -S build it explicitly adds in the libraries and .o
> files necessary for linking executables.
>
> That expects the compiler binary to be in "bin" and looks for ../include
> and ../cc/include for headers, and ../lib and ../cc/lib for libraries.
> In the case of linking in libc, it looks for the name "libc.so" which
> should be symlinked to the C library you want to use.
>
> > Therefore, I think these steps would be needed:
> >
> > 1. download.sh to get GCC sources.
>
> You can just populate the packages directory yourself and skip
> download.sh if you like.  But sure, modifying that would work.
>
> > 2. patch them with android specific headers and implementation
>
> I note that sources/patches exists to apply patches to upstream
> tarballs.  The extract logic will automatically apply patches with the
> right names ("packagename-*.patch", the required dash is so uClibc
> doesn't get uClibc++ patches applied to it).
>
> > 3. native-build.sh for ARM
>
> In theory that code is completely target-agnostic.  (And the actual
> individual package builds are in sources/sections so they can be re-used
> by the various stages as necessary.)
>
> In practice it's not necessarily _version_ agnostic: gcc 4.2 was just
> two packages, gcc 4.3 got split into four packages, and gcc 4.5 is
> written in C++ so requires a different host compiler setup.
>
> (The people who wrote gcc are completely insane, I'm really looking
> forward to llvm or pcc or something becoming usable to build Linux
> systems with.)
>
> > Next, I need to optimized for space, I need it as small as possible.
> > Therefore, disable CPP (with the given option),
>
> Actually cpp is the C preprocessor (I.E. what gcc -E calls out to), you
> want to disable c++.
>
> > remove pthreads, binutils etc etc -> but how do I config this?
>
> You can't remove binutils, it's the linker and the assembler.  (No, I
> don't know why the gcc developers split the linker and assembler out
> into a separate package.  I don't know why the split out GMP and MPFR in
> 4.3 either, but since it's GPLv3 I'm not comfortable shipping binaries
> built from it so it hasn't come up.  I note that Linux From Scratch has
> instructions for building those...)
>
> > I basically need to be really lean and mean. C compiler, libc, libm,
> > linux headers. thats it.
>
> Have you looked at tinycc?  It's a working C compiler (including linker
> and assembler all in one binary, busybox-style) in 100k.  Last I checked
> they claimed to have basic arm support working.  (Arm on an arm host
> didn't work for me, but that was 3 years ago...)
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110716/1baf0f6c/attachment.htm>

From rob at landley.net  Sat Jul 16 16:27:18 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 16 Jul 2011 11:27:18 -0500
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
Message-ID: <4E21BBE6.8070305@landley.net>

On 07/16/2011 07:53 AM, Earlence Fernandes wrote:
> Ok, I;ve built from source, the native compiler for armv5l. It compiles
> a couple of files I need on the devices perfectly.
> However, One particular file, I use netlink sockets and this is the
> message I get:
> 
> /data/local/tmp/narm/bin/../include/linux/netlink.h:31: error: expected
> specifier-qualifier-list before 'sa_family_t'
> 
> gb.c:310: error: 'struct sockaddr_nl' has no member named 'nl_family'
> gb.c:314: error: 'struct sockaddr_nl' has no member named 'nl_pid'

Hmmm...  either libc version skew or kernel version skew.  The headers
the package is building against aren't providing what it expects.

Let's see, ctrl-alt-google and:

  https://bugzilla.redhat.com/show_bug.cgi?id=561687

It looks like you need to #include <bits/sockaddr.h> before the #include
linux/netlink.h.  (Probably in your package?)

http://groups.google.com/group/linux.kernel/browse_thread/thread/6de65a3145007ae5

That implies the problem is in strace.  I'm already patching strace to
build natively on armv5l, you can grab that binary from
http://landley.net/aboriginal/downloads/binaries/extras/strace-armv5l

And the strace source patches are under:

http://landley.net/hg/control-images/file/tip/images/static-tools/patches/

I'm in the process of writing a control-images howto. :)

Rob

 1310833638.0

From rob at landley.net  Sat Jul 16 17:16:31 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 16 Jul 2011 12:16:31 -0500
Subject: [Aboriginal] "sh: gettext: command not found" during host compile
In-Reply-To: <4E1DEE31.1090803@sky.com>
References: <4E194E9B.6010209@sky.com>	<4E1AE023.6010402@landley.net>	<4E1B4249.1030005@sky.com>	<4E1B942C.6050004@landley.net>
	<4E1CA83F.3040700@sky.com>	<4E1D940A.3020504@landley.net>
	<4E1DEE31.1090803@sky.com>
Message-ID: <4E21C76F.9080200@landley.net>

On 07/13/2011 02:12 PM, Matthew J Fletcher wrote:
> On 13/07/11 13:48, Rob Landley wrote:
>> On 07/12/2011 03:02 PM, Matthew J Fletcher wrote:
>>>> Ubuntu turned gcc into a perl script that calls gcc.  (Yes really.)
>>>> Sounds like Mandrake might have done the same thing, if so I'll have to
>>>> install a copy here and fix it...
>>>
>>> On this box /usr/bin/gcc which simlinks to cc is a binary, nano says the
>>> first bytes are ^?ELF.
>>
>> So that's not it...
>>
>> I note that leaving mandrivia running overnight trying to format the
>> virtual hard drive didn't complete.  Sigh...
> 
> Used Virtualbox for years, works find with pretty much all the distros
> and Windows versions i have tried. Mandriva included.

I tried qemu built from source, and the kvm that came with ubuntu 10.04.
 Both hung at the same point.  Maybe 512 megs isn't enough memory?  (My
laptop gets unhappy in a swappy way when I give it a gig, to the point I
can't click the darn buttons because the mouse keeps overshooting them.
 The default mandrake mouse sensitivity is INSANELY high...)

it's too bad this 4 gigabyte dvd image doesn't seem to have a "live"
mode.  I'd download another cd image to try out an alternate version,
but I can't find any CD images, and this dvd image took all night to load.

I've got virtualbox on the windows machine at work, maybe I'll stay late
some day and try mandrivia there...

Rob

 1310836591.0

From earlenceferns at gmail.com  Sat Jul 16 17:16:34 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Sat, 16 Jul 2011 19:16:34 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <4E21BBE6.8070305@landley.net>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
Message-ID: <CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>

ok. doing what was suggested in that link got it compiled.

I compile my file like the following:

gcc -static gb.c -ldl

however, when I run it, it segfaults in the call to dlopen

This is the line in the file:

void *dlh = dlopen("/system/lib/libc.so", RTLD_NOW);

-Earlence

On Sat, Jul 16, 2011 at 6:27 PM, Rob Landley <rob at landley.net> wrote:

> On 07/16/2011 07:53 AM, Earlence Fernandes wrote:
> > Ok, I;ve built from source, the native compiler for armv5l. It compiles
> > a couple of files I need on the devices perfectly.
> > However, One particular file, I use netlink sockets and this is the
> > message I get:
> >
> > /data/local/tmp/narm/bin/../include/linux/netlink.h:31: error: expected
> > specifier-qualifier-list before 'sa_family_t'
> >
> > gb.c:310: error: 'struct sockaddr_nl' has no member named 'nl_family'
> > gb.c:314: error: 'struct sockaddr_nl' has no member named 'nl_pid'
>
> Hmmm...  either libc version skew or kernel version skew.  The headers
> the package is building against aren't providing what it expects.
>
> Let's see, ctrl-alt-google and:
>
>  https://bugzilla.redhat.com/show_bug.cgi?id=561687
>
> It looks like you need to #include <bits/sockaddr.h> before the #include
> linux/netlink.h.  (Probably in your package?)
>
>
> http://groups.google.com/group/linux.kernel/browse_thread/thread/6de65a3145007ae5
>
> That implies the problem is in strace.  I'm already patching strace to
> build natively on armv5l, you can grab that binary from
> http://landley.net/aboriginal/downloads/binaries/extras/strace-armv5l
>
> And the strace source patches are under:
>
> http://landley.net/hg/control-images/file/tip/images/static-tools/patches/
>
> I'm in the process of writing a control-images howto. :)
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110716/14e7a119/attachment.htm>

From rob at landley.net  Sat Jul 16 17:30:19 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 16 Jul 2011 12:30:19 -0500
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00R0-cFZcBVZZJg92_bfPkXH3xFccNNva+D78Kf_FMywgw@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<4E21B793.1030501@landley.net>
	<CA+ve00R0-cFZcBVZZJg92_bfPkXH3xFccNNva+D78Kf_FMywgw@mail.gmail.com>
Message-ID: <4E21CAAB.9060803@landley.net>

On 07/16/2011 11:15 AM, Earlence Fernandes wrote:
>>Have you looked at tinycc? 
> Yes, but theres not much development going on there and theres a lack of
> support on the mailing lists.

My own experiences with it are summarized at
http://landley.net/code/tinycc

And just recently:
http://lists.gnu.org/archive/html/tinycc-devel/2011-07/msg00023.html

> As I said, I managed to build it around 35MB (is fine for me) and the
> basic "hello world" works on the device.

Yay!

> I also patched in code for android that I need (like
> system_properties.c) (and it works)
> 
> The only error I'm getting now is ->
> 
> /data/local/tmp/narm/bin/../include/linux/netlink.h:40: error: expected
> specifier-qualifier-list before 'sa_family_t'
> gb.c: In function 'last_try':
> gb.c:310: error: 'struct sockaddr_nl' has no member named 'nl_family'
> gb.c:314: error: 'struct sockaddr_nl' has no member named 'nl_pid'
> gb.c: In function 'do_fault':
> gb.c:347: error: 'struct sockaddr_nl' has no member named 'nl_family'
> gb.c:351: error: 'struct sockaddr_nl' has no member named 'nl_pid'
> 
> I suspect this is not android specific. Can you help me out here?

I believe our emails are crossing paths, so I'll go read your next
message now. :)

> -Earlence

Rob

 1310837419.0

From rob at landley.net  Sat Jul 16 17:42:44 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 16 Jul 2011 12:42:44 -0500
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
Message-ID: <4E21CD94.6060406@landley.net>

On 07/16/2011 12:16 PM, Earlence Fernandes wrote:
> ok. doing what was suggested in that link got it compiled.
> 
> I compile my file like the following:
> 
> gcc -static gb.c -ldl

Would I be familiar with this gb.c software, or is it home-grown?

> however, when I run it, it segfaults in the call to dlopen
> 
> This is the line in the file:
> 
> void *dlh = dlopen("/system/lib/libc.so", RTLD_NOW);

Hmmm...  A statically linked uClibc program calling a dlopen() is
sufficiently black magic that I'd ask the uClibc developers on their
mailing list.  (You're using a dynamic linker that isn't loaded, there's
sort of a chicken and egg problem accessing it...)

A quick google finds:

  http://lists.uclibc.org/pipermail/uclibc/2005-July/033038.html
  http://lists.uclibc.org/pipermail/uclibc-cvs/2005-July/019085.html

Which suggests it was fixed ages ago and should work now, but there
might have been a regression...

I know that this _should_ work on an android kernel, people use android
kernels with glibc userspace all the time so no reason it shouldn't work
with uClibc.  So it's 99% likely to be just a uClibc bug.

One thing to try is using the current 0.9.32 release of uClibc.  I just
put a temporary copy of that at:

  http://landley.net/native-compiler-armv5l.tar.bz2

I note this hasn't been nearly as tested as the 0.9.31 version I'm
using.  That's their new release that came out recently.  (It includes
NTPL support, although I believe I've compiled it for pthreads at the
moment.)

If that works for you, it should be in the next release (aiming for the
end of the month).

Another thing to try is installing the shared libraries on the android
host, in their own directory, and doing a dynamically linked version.
That's a touch more complicated though.  (I have support for doing this
at non-standard paths, but don't remember the name of the control knobs
off the top of my head and it's been a while since I tested it.  I can
look it up if you're interested and write up documentation on how to do
it.  Fixing the static linking bug might be easier, especially if
there's an existing patch or easy workaround...)

> -Earlence

Rob

 1310838164.0

From earlenceferns at gmail.com  Sat Jul 16 17:59:20 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Sat, 16 Jul 2011 19:59:20 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <4E21CD94.6060406@landley.net>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
	<4E21CD94.6060406@landley.net>
Message-ID: <CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>

Thank you for the reply.

1. gb is custom software (sorry I should have made a note of that).
2. I rewrote a simple program with one line

void *dlh = dlopen("/system/lib/libc.so", RTLD_NOW);

I tried it on my build - same problem. seg fault.
I tried it with the build you sent me, same problem - seg faults again.

Is it possible to write a program precompiled with Android's dynamic linker
that will bootstrap the uclibc version? (I know going off topic here).
Or would it be possible to link the program against android ldl.a in the
first place?

-Earlence

On Sat, Jul 16, 2011 at 7:42 PM, Rob Landley <rob at landley.net> wrote:

> On 07/16/2011 12:16 PM, Earlence Fernandes wrote:
> > ok. doing what was suggested in that link got it compiled.
> >
> > I compile my file like the following:
> >
> > gcc -static gb.c -ldl
>
> Would I be familiar with this gb.c software, or is it home-grown?
>
> > however, when I run it, it segfaults in the call to dlopen
> >
> > This is the line in the file:
> >
> > void *dlh = dlopen("/system/lib/libc.so", RTLD_NOW);
>
> Hmmm...  A statically linked uClibc program calling a dlopen() is
> sufficiently black magic that I'd ask the uClibc developers on their
> mailing list.  (You're using a dynamic linker that isn't loaded, there's
> sort of a chicken and egg problem accessing it...)
>
> A quick google finds:
>
>  http://lists.uclibc.org/pipermail/uclibc/2005-July/033038.html
>  http://lists.uclibc.org/pipermail/uclibc-cvs/2005-July/019085.html
>
> Which suggests it was fixed ages ago and should work now, but there
> might have been a regression...
>
> I know that this _should_ work on an android kernel, people use android
> kernels with glibc userspace all the time so no reason it shouldn't work
> with uClibc.  So it's 99% likely to be just a uClibc bug.
>
> One thing to try is using the current 0.9.32 release of uClibc.  I just
> put a temporary copy of that at:
>
>  http://landley.net/native-compiler-armv5l.tar.bz2
>
> I note this hasn't been nearly as tested as the 0.9.31 version I'm
> using.  That's their new release that came out recently.  (It includes
> NTPL support, although I believe I've compiled it for pthreads at the
> moment.)
>
> If that works for you, it should be in the next release (aiming for the
> end of the month).
>
> Another thing to try is installing the shared libraries on the android
> host, in their own directory, and doing a dynamically linked version.
> That's a touch more complicated though.  (I have support for doing this
> at non-standard paths, but don't remember the name of the control knobs
> off the top of my head and it's been a while since I tested it.  I can
> look it up if you're interested and write up documentation on how to do
> it.  Fixing the static linking bug might be easier, especially if
> there's an existing patch or easy workaround...)
>
> > -Earlence
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110716/d46adb73/attachment.htm>

From rob at landley.net  Sat Jul 16 18:18:05 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 16 Jul 2011 13:18:05 -0500
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
	<4E21CD94.6060406@landley.net>
	<CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>
Message-ID: <4E21D5DD.2070902@landley.net>

On 07/16/2011 12:59 PM, Earlence Fernandes wrote:
> Thank you for the reply.
> 
> 1. gb is custom software (sorry I should have made a note of that).
> 2. I rewrote a simple program with one line
> 
> void *dlh = dlopen("/system/lib/libc.so", RTLD_NOW);

Um, question: is that bionic's libc you're attempting to dlopen?  (It's
an ELF binary, right?  What does readelf -a say about it?  Note that
attemting to dlopen() a linker script may not end well.)

Have you tried to dlopen() one of the other shared libraries that comes
with uClibc just to make sure that the issue is dlopen() and not it
being really confused by that particular library?

> I tried it on my build - same problem. seg fault.
> I tried it with the build you sent me, same problem - seg faults again.
> 
> Is it possible to write a program precompiled with Android's dynamic
> linker that will bootstrap the uclibc version? (I know going off topic
> here).

I don't understand what you're trying to do there, could you explain?
(You can exec one program from another, if that's what you mean by
boostrap.)

> Or would it be possible to link the program against android ldl.a in the
> first place?

I expect it's possible to link it, whether the result would work
involves knowing more about android than I do.

> -Earlence

Rob

 1310840285.0

From earlenceferns at gmail.com  Sat Jul 16 18:49:18 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Sat, 16 Jul 2011 20:49:18 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <4E21D5DD.2070902@landley.net>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
	<4E21CD94.6060406@landley.net>
	<CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>
	<4E21D5DD.2070902@landley.net>
Message-ID: <CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>

Yes, it is bionic's libc.
This is what readelf says about the header:
$ readelf -h /system/lib/libc.so
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          52 (bytes into file)
  Start of section headers:          273024 (bytes into file)
  Flags:                             0x5000000, Version5 EABI
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         6
  Size of section headers:           40 (bytes)
  Number of section headers:         22
  Section header string table index: 21

therefore, We can conclude
1. its a proper library for ARM EABI
2. its no an issue specific to my build (bcoz of the simple program I wrote
to read this libc.so and the result of running it with multiple builds).
3. its a ucLibC thing.

What I'm trying to do is basically compile and execute one of the root
exploits directly on the device. No! I'm not a malware writer, but we are
security researchers (http://few.vu.nl/~earlence)

http://pastebin.com/JTUbLjcz
That is the GingerBreak exploit that I'm trying to compile.

-Earlence

On Sat, Jul 16, 2011 at 8:18 PM, Rob Landley <rob at landley.net> wrote:

> On 07/16/2011 12:59 PM, Earlence Fernandes wrote:
> > Thank you for the reply.
> >
> > 1. gb is custom software (sorry I should have made a note of that).
> > 2. I rewrote a simple program with one line
> >
> > void *dlh = dlopen("/system/lib/libc.so", RTLD_NOW);
>
> Um, question: is that bionic's libc you're attempting to dlopen?  (It's
> an ELF binary, right?  What does readelf -a say about it?  Note that
> attemting to dlopen() a linker script may not end well.)
>
> Have you tried to dlopen() one of the other shared libraries that comes
> with uClibc just to make sure that the issue is dlopen() and not it
> being really confused by that particular library?
>
> > I tried it on my build - same problem. seg fault.
> > I tried it with the build you sent me, same problem - seg faults again.
> >
> > Is it possible to write a program precompiled with Android's dynamic
> > linker that will bootstrap the uclibc version? (I know going off topic
> > here).
>
> I don't understand what you're trying to do there, could you explain?
> (You can exec one program from another, if that's what you mean by
> boostrap.)
>
> > Or would it be possible to link the program against android ldl.a in the
> > first place?
>
> I expect it's possible to link it, whether the result would work
> involves knowing more about android than I do.
>
> > -Earlence
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110716/8b8e9a22/attachment.htm>

From earlenceferns at gmail.com  Sat Jul 16 19:05:40 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Sat, 16 Jul 2011 21:05:40 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
	<4E21CD94.6060406@landley.net>
	<CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>
	<4E21D5DD.2070902@landley.net>
	<CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>
Message-ID: <CA+ve00Rb+hB65WR2-yp-3RVPdiH9G9a82DJPO3ZhEeYt8+zriQ@mail.gmail.com>

>Have you tried to dlopen() one of the other shared libraries that comes
>with uClibc just to make sure that the issue is dlopen() and not it
>being really confused by that particular library?

I tried it with libm.a shipped with my native compiler. It worked ok!!

strange that is doesn't open /system/lib/libc.so
Thoughts?

-Earlence

On Sat, Jul 16, 2011 at 8:49 PM, Earlence Fernandes <earlenceferns at gmail.com
> wrote:

> Yes, it is bionic's libc.
> This is what readelf says about the header:
> $ readelf -h /system/lib/libc.so
> ELF Header:
>   Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
>   Class:                             ELF32
>   Data:                              2's complement, little endian
>   Version:                           1 (current)
>   OS/ABI:                            UNIX - System V
>   ABI Version:                       0
>   Type:                              DYN (Shared object file)
>   Machine:                           ARM
>   Version:                           0x1
>   Entry point address:               0x0
>   Start of program headers:          52 (bytes into file)
>   Start of section headers:          273024 (bytes into file)
>   Flags:                             0x5000000, Version5 EABI
>   Size of this header:               52 (bytes)
>   Size of program headers:           32 (bytes)
>   Number of program headers:         6
>   Size of section headers:           40 (bytes)
>   Number of section headers:         22
>   Section header string table index: 21
>
> therefore, We can conclude
> 1. its a proper library for ARM EABI
> 2. its no an issue specific to my build (bcoz of the simple program I wrote
> to read this libc.so and the result of running it with multiple builds).
> 3. its a ucLibC thing.
>
> What I'm trying to do is basically compile and execute one of the root
> exploits directly on the device. No! I'm not a malware writer, but we are
> security researchers (http://few.vu.nl/~earlence)
>
> http://pastebin.com/JTUbLjcz
> That is the GingerBreak exploit that I'm trying to compile.
>
> -Earlence
>
>
> On Sat, Jul 16, 2011 at 8:18 PM, Rob Landley <rob at landley.net> wrote:
>
>> On 07/16/2011 12:59 PM, Earlence Fernandes wrote:
>> > Thank you for the reply.
>> >
>> > 1. gb is custom software (sorry I should have made a note of that).
>> > 2. I rewrote a simple program with one line
>> >
>> > void *dlh = dlopen("/system/lib/libc.so", RTLD_NOW);
>>
>> Um, question: is that bionic's libc you're attempting to dlopen?  (It's
>> an ELF binary, right?  What does readelf -a say about it?  Note that
>> attemting to dlopen() a linker script may not end well.)
>>
>> Have you tried to dlopen() one of the other shared libraries that comes
>> with uClibc just to make sure that the issue is dlopen() and not it
>> being really confused by that particular library?
>>
>> > I tried it on my build - same problem. seg fault.
>> > I tried it with the build you sent me, same problem - seg faults again.
>> >
>> > Is it possible to write a program precompiled with Android's dynamic
>> > linker that will bootstrap the uclibc version? (I know going off topic
>> > here).
>>
>> I don't understand what you're trying to do there, could you explain?
>> (You can exec one program from another, if that's what you mean by
>> boostrap.)
>>
>> > Or would it be possible to link the program against android ldl.a in the
>> > first place?
>>
>> I expect it's possible to link it, whether the result would work
>> involves knowing more about android than I do.
>>
>> > -Earlence
>>
>> Rob
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110716/0dc46146/attachment.htm>

From earlenceferns at gmail.com  Sat Jul 16 19:16:36 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Sat, 16 Jul 2011 21:16:36 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00Rb+hB65WR2-yp-3RVPdiH9G9a82DJPO3ZhEeYt8+zriQ@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
	<4E21CD94.6060406@landley.net>
	<CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>
	<4E21D5DD.2070902@landley.net>
	<CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>
	<CA+ve00Rb+hB65WR2-yp-3RVPdiH9G9a82DJPO3ZhEeYt8+zriQ@mail.gmail.com>
Message-ID: <CA+ve00SEyC_W1keUaQa6JzzxL9TLj6juNzW19c3GSPy_CX9Z+A@mail.gmail.com>

note that I modify PATH and LD_LIBRARY_PATH to _include_ the bin/ and lib/
of the native compiler.
otherwise, the executable does not run.
and I specify an absolute path in dlopen, so I dont see how it can get
confused.

-Earlence

On Sat, Jul 16, 2011 at 9:05 PM, Earlence Fernandes <earlenceferns at gmail.com
> wrote:

> >Have you tried to dlopen() one of the other shared libraries that comes
> >with uClibc just to make sure that the issue is dlopen() and not it
> >being really confused by that particular library?
>
> I tried it with libm.a shipped with my native compiler. It worked ok!!
>
> strange that is doesn't open /system/lib/libc.so
> Thoughts?
>
> -Earlence
>
>
> On Sat, Jul 16, 2011 at 8:49 PM, Earlence Fernandes <
> earlenceferns at gmail.com> wrote:
>
>> Yes, it is bionic's libc.
>> This is what readelf says about the header:
>> $ readelf -h /system/lib/libc.so
>> ELF Header:
>>   Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
>>   Class:                             ELF32
>>   Data:                              2's complement, little endian
>>   Version:                           1 (current)
>>   OS/ABI:                            UNIX - System V
>>   ABI Version:                       0
>>   Type:                              DYN (Shared object file)
>>   Machine:                           ARM
>>   Version:                           0x1
>>   Entry point address:               0x0
>>   Start of program headers:          52 (bytes into file)
>>   Start of section headers:          273024 (bytes into file)
>>   Flags:                             0x5000000, Version5 EABI
>>   Size of this header:               52 (bytes)
>>   Size of program headers:           32 (bytes)
>>   Number of program headers:         6
>>   Size of section headers:           40 (bytes)
>>   Number of section headers:         22
>>   Section header string table index: 21
>>
>> therefore, We can conclude
>> 1. its a proper library for ARM EABI
>> 2. its no an issue specific to my build (bcoz of the simple program I
>> wrote to read this libc.so and the result of running it with multiple
>> builds).
>> 3. its a ucLibC thing.
>>
>> What I'm trying to do is basically compile and execute one of the root
>> exploits directly on the device. No! I'm not a malware writer, but we are
>> security researchers (http://few.vu.nl/~earlence)
>>
>> http://pastebin.com/JTUbLjcz
>> That is the GingerBreak exploit that I'm trying to compile.
>>
>> -Earlence
>>
>>
>> On Sat, Jul 16, 2011 at 8:18 PM, Rob Landley <rob at landley.net> wrote:
>>
>>> On 07/16/2011 12:59 PM, Earlence Fernandes wrote:
>>> > Thank you for the reply.
>>> >
>>> > 1. gb is custom software (sorry I should have made a note of that).
>>> > 2. I rewrote a simple program with one line
>>> >
>>> > void *dlh = dlopen("/system/lib/libc.so", RTLD_NOW);
>>>
>>> Um, question: is that bionic's libc you're attempting to dlopen?  (It's
>>> an ELF binary, right?  What does readelf -a say about it?  Note that
>>> attemting to dlopen() a linker script may not end well.)
>>>
>>> Have you tried to dlopen() one of the other shared libraries that comes
>>> with uClibc just to make sure that the issue is dlopen() and not it
>>> being really confused by that particular library?
>>>
>>> > I tried it on my build - same problem. seg fault.
>>> > I tried it with the build you sent me, same problem - seg faults again.
>>> >
>>> > Is it possible to write a program precompiled with Android's dynamic
>>> > linker that will bootstrap the uclibc version? (I know going off topic
>>> > here).
>>>
>>> I don't understand what you're trying to do there, could you explain?
>>> (You can exec one program from another, if that's what you mean by
>>> boostrap.)
>>>
>>> > Or would it be possible to link the program against android ldl.a in
>>> the
>>> > first place?
>>>
>>> I expect it's possible to link it, whether the result would work
>>> involves knowing more about android than I do.
>>>
>>> > -Earlence
>>>
>>> Rob
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110716/2a4eba8c/attachment.htm>

From rob at landley.net  Sun Jul 17 00:33:42 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 16 Jul 2011 19:33:42 -0500
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
	<4E21CD94.6060406@landley.net>
	<CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>
	<4E21D5DD.2070902@landley.net>
	<CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>
Message-ID: <4E222DE6.2050107@landley.net>

On 07/16/2011 01:49 PM, Earlence Fernandes wrote:
> Yes, it is bionic's libc.
> This is what readelf says about the header:
> $ readelf -h /system/lib/libc.so
> ELF Header:
>   Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
>   Class:                             ELF32
>   Data:                              2's complement, little endian
>   Version:                           1 (current)
>   OS/ABI:                            UNIX - System V
>   ABI Version:                       0
>   Type:                              DYN (Shared object file)
>   Machine:                           ARM
>   Version:                           0x1
>   Entry point address:               0x0
>   Start of program headers:          52 (bytes into file)
>   Start of section headers:          273024 (bytes into file)
>   Flags:                             0x5000000, Version5 EABI
>   Size of this header:               52 (bytes)
>   Size of program headers:           32 (bytes)
>   Number of program headers:         6
>   Size of section headers:           40 (bytes)
>   Number of section headers:         22
>   Section header string table index: 21
> 
> therefore, We can conclude
> 1. its a proper library for ARM EABI
> 2. its no an issue specific to my build (bcoz of the simple program I
> wrote to read this libc.so and the result of running it with multiple
> builds).
> 3. its a ucLibC thing.
> 
> What I'm trying to do is basically compile and execute one of the root
> exploits directly on the device. No! I'm not a malware writer, but we
> are security researchers (http://few.vu.nl/~earlence)
> 
> http://pastebin.com/JTUbLjcz
> That is the GingerBreak exploit that I'm trying to compile.
> 
> -Earlence

Alas, you're asking uClibc questions, which need to be asked on the
uClibc mailing list.

It's reasonably straightforward to drop the bionic libraries in place of
the uClibc libraries, the compiler itself should have no dependencies on
uClibc (with the possible exception of libgcc.so, but you can just
delete that and ccwrap will use libgcc.a and libgcc_eh.a instead).

A good start would be reading the ccwrap.c source, it's reasonably small
and overrides all of gcc's path logic that I could find.

Rob

 1310862822.0

From rob at landley.net  Sun Jul 17 00:34:45 2011
From: rob at landley.net (Rob Landley)
Date: Sat, 16 Jul 2011 19:34:45 -0500
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00SEyC_W1keUaQa6JzzxL9TLj6juNzW19c3GSPy_CX9Z+A@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
	<4E21CD94.6060406@landley.net>
	<CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>
	<4E21D5DD.2070902@landley.net>
	<CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>
	<CA+ve00Rb+hB65WR2-yp-3RVPdiH9G9a82DJPO3ZhEeYt8+zriQ@mail.gmail.com>
	<CA+ve00SEyC_W1keUaQa6JzzxL9TLj6juNzW19c3GSPy_CX9Z+A@mail.gmail.com>
Message-ID: <4E222E25.2010006@landley.net>

On 07/16/2011 02:16 PM, Earlence Fernandes wrote:
> note that I modify PATH and LD_LIBRARY_PATH to _include_ the bin/ and
> lib/ of the native compiler.
> otherwise, the executable does not run.
> and I specify an absolute path in dlopen, so I dont see how it can get
> confused.
> 
> -Earlence

You have the source to uClibc, you could always read the dlopen source
and see what it's doing.  (That's what I'd do if I had a test
environment for this...)

Rob

 1310862885.0

From earlenceferns at gmail.com  Sun Jul 17 06:46:20 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Sun, 17 Jul 2011 08:46:20 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <4E222E25.2010006@landley.net>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
	<4E21CD94.6060406@landley.net>
	<CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>
	<4E21D5DD.2070902@landley.net>
	<CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>
	<CA+ve00Rb+hB65WR2-yp-3RVPdiH9G9a82DJPO3ZhEeYt8+zriQ@mail.gmail.com>
	<CA+ve00SEyC_W1keUaQa6JzzxL9TLj6juNzW19c3GSPy_CX9Z+A@mail.gmail.com>
	<4E222E25.2010006@landley.net>
Message-ID: <CA+ve00TsyDwc97tz9524rsHbftyNPry=0azYJoEmmhwEQd1PGA@mail.gmail.com>

Yes, thanks, you have been very helpful.
I was able to get it working by using a hack where I got the address of the
function from libc.so from another cross-compiled program.
However, if I have to make this more stable, I will have to see whats going
wrong with dlopen.

-Earlence

On Sun, Jul 17, 2011 at 2:34 AM, Rob Landley <rob at landley.net> wrote:

> On 07/16/2011 02:16 PM, Earlence Fernandes wrote:
> > note that I modify PATH and LD_LIBRARY_PATH to _include_ the bin/ and
> > lib/ of the native compiler.
> > otherwise, the executable does not run.
> > and I specify an absolute path in dlopen, so I dont see how it can get
> > confused.
> >
> > -Earlence
>
> You have the source to uClibc, you could always read the dlopen source
> and see what it's doing.  (That's what I'd do if I had a test
> environment for this...)
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110717/f3a43f79/attachment.htm>

From rob at landley.net  Mon Jul 18 03:02:09 2011
From: rob at landley.net (Rob Landley)
Date: Sun, 17 Jul 2011 22:02:09 -0500
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <CA+ve00Rb+hB65WR2-yp-3RVPdiH9G9a82DJPO3ZhEeYt8+zriQ@mail.gmail.com>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>	<4E21BBE6.8070305@landley.net>	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>	<4E21CD94.6060406@landley.net>	<CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>	<4E21D5DD.2070902@landley.net>	<CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>
	<CA+ve00Rb+hB65WR2-yp-3RVPdiH9G9a82DJPO3ZhEeYt8+zriQ@mail.gmail.com>
Message-ID: <4E23A231.3020505@landley.net>

On 07/16/2011 02:05 PM, Earlence Fernandes wrote:
>>Have you tried to dlopen() one of the other shared libraries that comes
>>with uClibc just to make sure that the issue is dlopen() and not it
>>being really confused by that particular library?
> 
> I tried it with libm.a shipped with my native compiler. It worked ok!!
> 
> strange that is doesn't open /system/lib/libc.so
> Thoughts?
> 
> -Earlence

Weird, .a files are static libraries, not dynamic libraries.  I didn't
think dlopen could do anything useful with them.  Huh.

Shared libraries have all sorts of funky hooks and things, initializion
code, different types of relocations (REL vs RELA, lazy binding needs
its own extra set of tables, and of course the gnu guys extended elf
with their own hash stuff as blogged about
http://blogs.oracle.com/ali/entry/gnu_hash_elf_sections ...

There's a marvelous set of blog entries on writing a linker by the guy
who did GOLD, it's linked to as reference #1 from the gold wikipedia entry:

  http://en.wikipedia.org/wiki/Gold_(linker)

Denys Vlasenko did a marvelous talk on some linker subtleties he
encountered, the video of which is one of these:

  http://free-electrons.com/blog/elc-2010-videos/

If you want to learn about how linkers work, the standard reference is:

  http://www.iecc.com/linker/

Plus the ELF spec:

  http://refspecs.freestandards.org/elf/

So it's entirely possible something is subtly wrong with that .so
because they wrote their own dynamic linker that doesn't use all the
fields the others do, and thus uClibc's following an unexpected null
pointer or something.  But that's just a guess.

Rob

 1310958129.0

From earlenceferns at gmail.com  Mon Jul 18 04:19:16 2011
From: earlenceferns at gmail.com (Earlence Fernandes)
Date: Mon, 18 Jul 2011 06:19:16 +0200
Subject: [Aboriginal] Using native-build.sh for An Android native C
	compiler
In-Reply-To: <4E23A231.3020505@landley.net>
References: <CA+ve00Q1Ca4iF1vuMc_Vye9ctmkjJQWaWJbdfmT_JSVZBJaNrw@mail.gmail.com>
	<CA+ve00RNpiyGT698RvBbHw2aDKZN0oNUypRV5sLWCVNJyGSi5g@mail.gmail.com>
	<4E21BBE6.8070305@landley.net>
	<CA+ve00SFKTS51iQ0Gc9XEqzL8SEECQHM17wfMrh9UP0F6+vaYA@mail.gmail.com>
	<4E21CD94.6060406@landley.net>
	<CA+ve00TBGxOi9hxkgFVGfDDF0UoaOe=DEHFmYPTkq0=FfbzGSw@mail.gmail.com>
	<4E21D5DD.2070902@landley.net>
	<CA+ve00SLOorgboyqf3DPHcZYFh-0gOr367RrBcSfAZ_LusbdQw@mail.gmail.com>
	<CA+ve00Rb+hB65WR2-yp-3RVPdiH9G9a82DJPO3ZhEeYt8+zriQ@mail.gmail.com>
	<4E23A231.3020505@landley.net>
Message-ID: <CA+ve00R0OcrpB1BbaqmkoTSjj4O=nOtc9+ma2VMFzcyPXYvdQg@mail.gmail.com>

>Weird, .a files are static libraries, not dynamic libraries.  I didn't
>think dlopen could do anything useful with them.  Huh.

sorry! typo! I meant libm.so :)

Thanks for the other pointers, I'll read them'

-Earlence

On Mon, Jul 18, 2011 at 5:02 AM, Rob Landley <rob at landley.net> wrote:

> On 07/16/2011 02:05 PM, Earlence Fernandes wrote:
> >>Have you tried to dlopen() one of the other shared libraries that comes
> >>with uClibc just to make sure that the issue is dlopen() and not it
> >>being really confused by that particular library?
> >
> > I tried it with libm.a shipped with my native compiler. It worked ok!!
> >
> > strange that is doesn't open /system/lib/libc.so
> > Thoughts?
> >
> > -Earlence
>
> Weird, .a files are static libraries, not dynamic libraries.  I didn't
> think dlopen could do anything useful with them.  Huh.
>
> Shared libraries have all sorts of funky hooks and things, initializion
> code, different types of relocations (REL vs RELA, lazy binding needs
> its own extra set of tables, and of course the gnu guys extended elf
> with their own hash stuff as blogged about
> http://blogs.oracle.com/ali/entry/gnu_hash_elf_sections ...
>
> There's a marvelous set of blog entries on writing a linker by the guy
> who did GOLD, it's linked to as reference #1 from the gold wikipedia entry:
>
>  http://en.wikipedia.org/wiki/Gold_(linker)
>
> Denys Vlasenko did a marvelous talk on some linker subtleties he
> encountered, the video of which is one of these:
>
>  http://free-electrons.com/blog/elc-2010-videos/
>
> If you want to learn about how linkers work, the standard reference is:
>
>  http://www.iecc.com/linker/
>
> Plus the ELF spec:
>
>  http://refspecs.freestandards.org/elf/
>
> So it's entirely possible something is subtly wrong with that .so
> because they wrote their own dynamic linker that doesn't use all the
> fields the others do, and thus uClibc's following an unexpected null
> pointer or something.  But that's just a guess.
>
> Rob
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.landley.net/pipermail/aboriginal/attachments/20110718/513a6517/attachment.htm>

From rob at landley.net  Tue Jul 19 12:49:32 2011
From: rob at landley.net (Rob Landley)
Date: Tue, 19 Jul 2011 07:49:32 -0500
Subject: [Aboriginal] First stab at a control-images page.
Message-ID: <4E257D5C.7070809@landley.net>

http://landley.net/code/control-images/

Let me know what you think.  I'm updating lfs-bootstrap to 6.8 and will
probably document what's involved with that as I do it.  Just need to
find the time...

(Yeah, I know the source and binaries links aren't there yet, I haven't
cut a release.)

Rob

 1311079772.0

From rob at landley.net  Tue Jul 19 23:56:50 2011
From: rob at landley.net (Rob Landley)
Date: Tue, 19 Jul 2011 18:56:50 -0500
Subject: [Aboriginal] Clang builds Linux kernel
In-Reply-To: <CAELZRKYM6o2TEhepFuqn332bKwBt-Jk=h+EU1sZRqiYoVdPkxg@mail.gmail.com>
References: <CAELZRKYM6o2TEhepFuqn332bKwBt-Jk=h+EU1sZRqiYoVdPkxg@mail.gmail.com>
Message-ID: <4E2619C2.6000604@landley.net>

On 07/19/2011 12:37 PM, prasanna balan wrote:
> Hello,
>          I'm prazz from aboriginal mailing lists........Just kept it private
> 
> I bumped upon this article moments ago
> 
> http://lists.cs.uiuc.edu/pipermail/cfe-dev/2010-October/011711.html
> 
> It tells about llvm/Clang compiling the kernel....hurray
> 
> Need to test it with aboriginal
> 
> prazz

Yay!

I'll put it on my todo list for this weekend.

Thanks,

Rob



 1311119810.0

From rob at landley.net  Thu Jul 21 00:42:27 2011
From: rob at landley.net (Rob Landley)
Date: Wed, 20 Jul 2011 19:42:27 -0500
Subject: [Aboriginal] Clang builds Linux kernel
In-Reply-To: <CAELZRKYM6o2TEhepFuqn332bKwBt-Jk=h+EU1sZRqiYoVdPkxg@mail.gmail.com>
References: <CAELZRKYM6o2TEhepFuqn332bKwBt-Jk=h+EU1sZRqiYoVdPkxg@mail.gmail.com>
Message-ID: <4E2775F3.5050106@landley.net>

On 07/19/2011 12:37 PM, prasanna balan wrote:
> Hello,
>          I'm prazz from aboriginal mailing lists........Just kept it private
> 
> I bumped upon this article moments ago
> 
> http://lists.cs.uiuc.edu/pipermail/cfe-dev/2010-October/011711.html
> 
> It tells about llvm/Clang compiling the kernel....hurray
> 
> Need to test it with aboriginal
> 
> prazz
> 

I remember now that I saw that when it first came out.  Unfortunately it
only supported x86 and only a subset of the kernel.

However, there has been some progress since then:

  http://lwn.net/Articles/441018/

Rob

 1311208947.0

From rob at landley.net  Sun Jul 24 18:56:14 2011
From: rob at landley.net (Rob Landley)
Date: Sun, 24 Jul 2011 13:56:14 -0500
Subject: [Aboriginal] Package upgrades and cleanup work.
Message-ID: <4E2C6ACE.1000906@landley.net>

1) Linux 3.0 is out: it broke busybox.

Fixes (to busybox):

A) rip out the redundant sysinfo definition busybox block copied out of
stale header files.
B) Zap two applets Denys inexplicably added which reproduce netcat
server mode.

I also needed to update the noperl patches, not because the perl removal
changed but because the context did.

2) Updating to uClibc 0.9.32: sh4 and sparc broke.

Fixing it:

  http://lists.uclibc.org/pipermail/uclibc/2011-July/045590.html

Still using pthreads.old rather than nptl, I'll worry about that later.

3) Changed the boot scripts to mount devtmpfs on /dev instead of
mounting a tmpfs there and running mdev -s.  (This is simpler, and gives
us free hotplug support.  The permissions only allow root to access most
stuff, but mdev is like that with no config file.)

4) Ripped out the control-images from the aboriginal repo (now that
they're in their own repo).  Might have missed something or left some
unconnected bits, but I think I got it all...

Vaguely on track for a release at the end of the month, I think...

Rob

Rob

 1311533774.0

From rob at landley.net  Wed Jul 27 18:57:37 2011
From: rob at landley.net (Rob Landley)
Date: Wed, 27 Jul 2011 13:57:37 -0500
Subject: [Aboriginal] Mips still needs the 2.6.39 fix patch.
Message-ID: <4E305FA1.5070404@landley.net>

That's why moving to Linux 3.0 broke it, fixed in the repo now.

(I've poked the mips guys as to why they didn't merge the darn patch
since they came up with it in May.  Haven't heard back yet...)

On-track-ish for a release next week.

Rob

 1311793057.0

From rob at landley.net  Sun Jul 31 16:42:34 2011
From: rob at landley.net (Rob Landley)
Date: Sun, 31 Jul 2011 11:42:34 -0500
Subject: [Aboriginal] Playing with other toolchains.
Message-ID: <4E3585FA.5090905@landley.net>

A lot of projects seem to use the prebuilt binary toolchain from the
Code Sourcery first quarter 2009 armv7 release, so let's download that:

http://www.codesourcery.com/sgpp/lite/arm/portal/package4571/public/arm-none-linux-gnueabi/arm-2009q1-203-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2

Next extract it and plug it into the build using the appropriate names:

cd aboriginal
tar xvjf ~/arm-2009q1*.tar.bz2
mkdir -p build/cross-compiler-armv7l/bin
for i in $(ls arm-2009q1/bin)
do
  RENAME="$(echo $i | sed 's/arm-none-linux-gnueabi/armv7l/')"
  ln -s ../../../arm-2009q1/bin/$i \
    build/cross-compiler-armv7l/bin/$RENAME
done
ln -s armv7l-gcc build/cross-compiler-armv7l/bin/armv7l-cc

Now let's slap together a quick and dirty config, changing the kernel's
V6 symbol for a V7 symbol:

  sed "s/V6/V7/" sources/targets/armv6l > sources/targets/armv7l

And let's try the build, using the existing toolchain:

  ./host-tools.sh
  ./simple-root-filesystem.sh armv7l

Alright, let's try...

  qemu-arm build/simple-root-filesystem-armv7l/bin/ash

Woot, we get a shell.  Exit out of that, second question: would it work
dynamically linked?  (Remember, busybox static by default...)

  readelf -a build/simple-root-filesystem-armv7l/bin/hello-dynamic | \
    grep interpreter

And it says:

  [Requesting program interpreter: /lib/ld-linux.so.3]

So:
  ls build/simple-root-filesystem-armv7l/lib/ld-linux.so.3

Well it's there.  Unfortunately, qemu-arm's -L is too fiddly to expect
much out of, and the -chroot option comes and goes (I submitted the
patch years ago, it was in there for a while, but I think you have to
enable it or soemthing).  So let's package it up for the muchy less
crappy system emulator:

  ./root-image.sh armv7l
  ./linux-kernel.sh armv7l
  ./system-image.sh armv7l

And the kernel build explodes.  At a guess, my versatile patch isn't
doing armv7 properly.  (Possibly I didn't bother to forward port that
since I had no way of building it.)

I need to grab a better linux-config.

Anybody else feel like playing with this?

Rob

 1312130554.0