[Toybox] Bad idea regarding threading...

[Elie De Brauwer]

On Mon, Apr 30, 2012 at 9:46 AM, Elie De Brauwer
<eliedebrauwer at gmail.com> wrote:
> On Mon, Apr 30, 2012 at 9:39 AM, David Seikel <onefang at gmail.com> wrote:
>> On Mon, 30 Apr 2012 09:25:37 +0200 Elie De Brauwer
>> <eliedebrauwer at gmail.com> wrote:
>>
>>> On Sun, Apr 29, 2012 at 2:57 AM, Rob Landley <rob at landley.net> wrote:
>>> > It occurs to me that if I add a CONFIG_TOYBOX_THREADS with the new
>>> > directory traversal infrastructure, things like cp -a and rm -r
>>> > could be done in a multithreaded manner.
>>> >
>>> > I.E. create a thread pool equal to the number of processors, and
>>> > then every time you encounter a directory hand off the callbacks to
>>> > a thread out of the thread pool. Everything they're doing is
>>> > openat() based on a filehandle stored in the node structure (or a
>>> > filehandle pair with the second stored in the node's ->extra
>>> > field), so you don't need to worry about the current directory
>>> > changing in another thread...
>>>
>>> Indeed, and I think that this could also be a very nice feature to
>>> differentiate toybox from similar tools, to my knowledge there aren't
>>> any 'userlands' availble which have inherent multithreading support
>>> (typically because most stem from the time that it was easier to
>>> purchase a human kidney than to purchase smp systems.
>>> The only thing I want to add there is that in such a scenario I expect
>>> the number of usable cores to be runtime configurable (e.g. through an
>>> environment variable).
>>
>> Um, I would hope that the code figures out for itself how many cores
>> are available.  Perhaps you meant artificially limiting cores, to keep
>> some in reserve for something else?
>>
>
> Indeed, when I have some cores dedicated to chewing some data, I don't
> want some recursive chmod which some **** typed into a prompt to flood
> the scheduler. So selecting the number of cores automatically is good,
> but allowing an override is better. And when you use magic such as
> isolcpus ( http://www.kernel.org/doc/Documentation/kernel-parameters.txt
> ) whilst  the number of cores is automatically selected, this will
> just result in a whole bunch of blocked threads. Anyhow, imho that all
> is.

As additional illustration, that is also how openmp functions, by
default it will create a thread per core, but it can be overridden
through environment variables ref
https://computing.llnl.gov/tutorials/openMP/#EnvironmentVariables

my 2 cents
E.

-- 
Elie De Brauwer

 1335772123.0