parasweep.dispatchers module¶
Dispatchers for running parallel jobs.
-
class
parasweep.dispatchers.
Dispatcher
[source]¶ Bases:
abc.ABC
Abstract base class for dispatchers.
Subclasses should implement the
initialize_session
,dispatch
, andwait_all
methods.
-
class
parasweep.dispatchers.
SubprocessDispatcher
(max_procs=None)[source]¶ Bases:
parasweep.dispatchers.Dispatcher
Dispatcher using subprocesses.
Parameters: max_procs (int, optional) – The maximum number of processes to run simultaneously. By default, uses the number of processors on the machine (this is a good choice for CPU-bound work).
-
class
parasweep.dispatchers.
DRMAADispatcher
(job_template=None)[source]¶ Bases:
parasweep.dispatchers.Dispatcher
Dispatcher for DRMAA.
Parameters: job_template (drmaa.JobTemplate instance, optional) – A job template containing settings for running the jobs with the job scheduler. Documentation for the different options is available in the Python drmaa package. Some options specific to each job scheduler, called the native specification, may have to be set using the job_template.nativeSpecification
attribute, the options for which can be found in the job scheduler’s DRMAA interface (e.g., slurm-drmaa for Slurm and pbs-drmaa for PBS).Examples
>>> import drmaa >>> jt = drmaa.JobTemplate(hardWallclockTimeLimit=60) >>> dispatcher = DRMAADispatcher(jt)
-
session
= None¶
-