Conda & Dask Service#
Conda & Dask Service Class#
Classes:
|
Deploy a Dask Cluster. |
- class enoslib.service.conda.conda.Dask(conda_env: str, conda_prefix: str | None = None, scheduler: Host | None = None, workers: Iterable[Host] | None = None, worker_args: str = '', run_as: str = 'root')#
Deploy a Dask Cluster.
It bootstraps a Dask scheduler and workers by activating the passed conda environment. User must have an environment ready with, at least, dask installed inside. The agents will be started as the passed user.
It can be used as a context manager. Note that the exit method isn’t optimal though (see
enoslib.service.conda.conda.AutoDask.destroy())- Parameters:
conda_env – name of the conda environment (on the remote system)
conda_prefix – prefix of the conda installation (will be used to bring conda in the env) Default to /home/<run_as>/miniconda3
scheduler – Host that will serve as the dask scheduler
workers – List of Host that will serve as workers
worker_args – specific worker args to pass (e.g “–nthreads 1 –nprocs 8”)
run_as – remote user to use. Conda must be available to this user.
Examples
- backup()#
(abstract) Backup the service.
- deploy()#
(abstract) Deploy the service.
- destroy()#
Destroy the dask cluster.
Note that client.shutdown() is much more efficient.
- in_conda_cmd(cmd: str) str#
Transforms a command to be executed in the context of the current conda env.
- Parameters:
cmd – the command string
- Returns:
The transformed command string prefixed by some other to activate the conda env.