Task farming¶
-
brian.tools.taskfarm.
run_tasks
(dataman, task, items, gui=True, poolsize=0, initargs=None, initkwds=None, verbose=None, numitems=None)¶ Run a series of tasks using multiple CPUs on a single computer.
Initialised with arguments:
dataman
- The
DataManager
object used to store the results in, see below. task
- The task function or class (see below).
items
- A sequence (e.g. list or iterator) of arguments to be passed to the task.
gui=True
- Whether or not to use a Tkinter based GUI to show progress and terminate the task run.
poolsize=0
- The number of CPUs to use. If the value is 0, use all available CPUs, if it is -1 use all but one CPU, etc.
initargs
,initkwds
- If
task
is a class, these are the initialisation arguments and keywords for the class. verbose=None
- Specify True or False to print out every progress message (defaults to False if the GUI is used, or True if not).
numitems=None
- For iterables (rather than fixed length sequences), if you specify the number of items, an estimate of the time remaining will be given.
The task (defined by a function or class, see below) will be called on each item in
items
, and the results saved todataman
. Results are stored in the format(key, val)
wherekey
is a unique but meaningless identifier. Results can be retrieved usingdataman.values()
or (for large data sets that should be iterated over)dataman.itervalues()
.The task can either be a function or a class. If it is a function, it will be called for each item in
items
. If the items are tuples, the function will be called with those tuples as arguments (e.g. if the item is(1,2,3)
the function will be called astask(1, 2, 3)
). If the task is a class, it can have an__init__
method that is called once for each process (each CPU) at the beginning of the task run. If the__init__
method has aprocess_number
argument, it will be passed an integer value from 0 tonumprocesses-1
giving the number of the process (note, this is not the process ID). The class should define a__call__
method that behaves the same as above fortask
being a function. In both cases (function or class), if the arguments include a keywordreport
then it will be passed a value that can be passed as thereport
keyword in Brian’srun()
function to give feedback on the simulation as it runs. Atask
function can also setself.taskname
as a string that will be displayed on the GUI to give additional information.Warning
On Windows, make sure that
run_tasks()
is only called from within a block such as:if __name__=='__main__': run_tasks(...)
Otherwise, the program will go into a recursive loop.
Note that this class only allows you to run tasks on a single computer, to distribute work over multiple computers, we suggest using Playdoh.