Global API#

Root module#

Functions:

`check`([platform_filter])	Check the status of EnOSlib.
`init_logging`([level])	Enable Rich display of log messages.

enoslib.check(platform_filter: List[str] | None = None)#

Check the status of EnOSlib.

This gives you a synthetic view of

the installed providers (dependency check)
the connectivity to the various providers (connectivity check)

The dependency check test which optional dependencies are installed with your EnOSlib version. But you can specify the providers you want to check if you want to The connectivity check if connection to the various providers’ infrastructure can be initiated. Since each provider setup is specific, this gives you a quick feedback on your local configuration.

Args: platform_filter: A list of specific checks to perform.

Example

```python # Perform all checks check()

# Perform specific checks check([‘check1’, ‘check2’]) ```

enoslib.init_logging(level=20, **kwargs)#

Enable Rich display of log messages.

kwargs: kwargs passed to RichHandler.

EnOSlib chooses some defaults for you: show_time=False,

Config module#

Manage a configuration for EnOSlib.

Functions:

set_config([g5k_cache, g5k_auto_jump, ...])

Set a specific config value.

Set a specific config value.

Parameters:

g5k_cache – True iff a cache must be used for HTTP request to the API Reasons to disable the cache is to workaround issues with concurrent access on NFS.
g5k_auto_jump – control auto-jump configuration None: auto-detect if the jump over the access machine is necessary True: force jump over the access machine False: disable the jump over the access machine (e.g when using the VPN)
display – In a Jupyter environment, display objects using an HTML representation
dump_results – dump the command result in a file
ansible_stdout – stdout Ansible callback to use
ansible_forks – change Ansible’s “forks” parameter (level of parallelization)

API module#

Need to run some actions on your nodes ? This module is tailored for this purpose.

Remote actions rely heavily on Ansible [1] used as a library through its python API. Among other things a convenient context manager is brought to the experimenter (see play_on) to run arbitrary Ansible code without leaving the python world.

These function can be fed with library-level objects (see objects) and are thus provider agnostic.

Classes:

`Results`([iterable])	Container for CommandResults
`actions`(*[, pattern_hosts, inventory_path, ...])	Context manager to run a set of remote actions on nodes

Functions:

`ensure_python3`([make_default])	Make sure python3 is installed on the remote nodes, and optionally make it the default.
`gather_facts`(*[, pattern_hosts, ...])	Gather facts about hosts.
`generate_inventory`(roles, networks, ...[, ...])	Generate an inventory file in the ini format.
`get_hosts`(roles[, pattern_hosts])	Get all the hosts matching the pattern.
`run`(cmd, roles, **kwargs)	Run command on some hosts
`run_ansible`(playbooks[, inventory_path, ...])	Run Ansible.
`run_command`(command, *[, pattern_hosts, ...])	Run a shell command on some remote hosts.
`run_play`(play_source, *[, inventory_path, ...])	Run a play.
`sync_info`()	Sync each host network information with their actual configuration
`wait_for`(roles[, retries, interval])	Wait for all the machines to be ready to run some commands.

class enoslib.api.Results(iterable=(), /)#

Container for CommandResult**s**

Running one (or more) command(s) on several hosts leads to multiple results to be gathered by EnOSlib. EnOSlib manage the results as a flat list of individual result (one per host and command) but allow for some filtering to be done.

Example with a single command:

result = en.run_command("date", roles=roles)
# print the stdout of command on host "foo-1"
print([res.stdout for res in result.filter(host="foo-1")])

# get the stderr of failed tasks on all hosts
[res.stderr for res in result.filter(status=enoslib.STATUS_FAILED)]

# get all unreachable hosts
[res.host for res in result.filter(status=enoslib.STATUS_UNREACHABLE)]

Example with multiple commands:

with en.actions(roles=roles) as a:
    a.apt(task_name="Install htop", name="htop", state="present")
    a.command(task_name="Get date", cmd="date")
    results = a.results

# print the stdout of "Get date" tasks on all hosts
print([res.stdout for res in result.filter(task="Get date")])

to_dict(include_payload: bool = False) → List[Dict]#

A dict representation of a Results

Use case: JSON serialization

Parameters:: include_payload – True wheter the raw payload must be included.
Returns:: A list of the dict, each element represents one Result.

class enoslib.api.actions(*, pattern_hosts: str = 'all', inventory_path: str | None = None, roles: Roles | Iterable[Host] | Host | None = None, gather_facts: bool = False, priors: List[actions] | None = None, run_as: str | None = None, background: bool = False, strategy: str = 'linear', **kwargs)#

Context manager to run a set of remote actions on nodes

Parameters:

pattern_hosts (str) – pattern to describe ansible hosts to target. see https://docs.ansible.com/ansible/latest/intro_patterns.html
inventory_path (str) – inventory to use
roles (RolesLike) – roles as returned by enoslib.infra.provider.Provider.init()
extra_vars (dict) – extra_vars to use
on_error_continue (bool) – don’t throw any exception in case a host is unreachable or the playbooks run with errors
gather_facts (bool) – whether facts will be gathered.
priors (list) – tasks in each prior will be prepended in the playbook
run_as (str) – A shortcut that injects become and become_user to each task. become* at the task level has the precedence over this parameter.
background (bool) – A shortcut that injects async=1year, poll=0 to run the commands in detached mode. Can be overridden at the task level.
strategy (str) – ansible execution strategy
kwargs – keyword arguments passed to enoslib.api.run_ansible().

Examples

Minimal snippet:

with actions(roles=roles) as t:
    t.apt(name=["curl", "git"], state="present")
    t.shell("which docker || (curl get.docker.com | sh)")
    t.docker_container(name="nginx", state="started")

Complete example with fact_gathering

import logging

import enoslib as en

en.init_logging(level=logging.INFO)
en.check()

provider_conf = {
    "resources": {
        "machines": [
            {
                "roles": ["control"],
                "flavour": "tiny",
                "number": 1,
            },
            {
                "roles": ["client"],
                "flavour": "tiny",
                "number": 1,
            },
        ],
        "networks": [{"roles": ["rn1"], "cidr": "172.16.0.1/16"}],
    }
}

conf = en.VagrantConf.from_dictionary(provider_conf)
provider = en.Vagrant(conf)

roles, networks = provider.init()

with en.actions(roles=roles) as p:
    p.debug(msg="{{ inventory_hostname  }}")

with en.actions(pattern_hosts="client", roles=roles) as p:
    p.debug(msg="{{ inventory_hostname  }}")

# Using the actions wrapper allows for using a list of hosts instead of a Roles object
with en.actions(roles=roles["client"]) as p:
    p.debug(msg="{{ inventory_hostname  }}")

with en.actions(roles=roles["client"], gather_facts=True) as p:
    p.debug(msg="{{ inventory_hostname  }}")


with en.actions(roles=roles["client"], gather_facts=False) as p:
    p.debug(msg="{{ inventory_hostname  }}")
    p.shell("sleep 3")
    p.shell("sleep 5", background=True)
    p.shell("sleep 3")


with en.actions(roles=roles["client"], gather_facts=False, background=True) as p:
    p.debug(msg="{{ inventory_hostname  }}", background=False)
    p.shell("sleep 3")
    p.shell("sleep 5", background=False)
    p.shell("sleep 3")

Hint

Module can be run asynchronously using the corresponding Ansible options. Note that not all the modules support asynchronous execution.
Note that the actual result isn’t available in the result file but will be available through a file specified in the result object.
Any ansible module can be called using the above way. You’ll need to refer to the module reference documentation to find the corresponding kwargs to use.

enoslib.api.ensure_python3(make_default: bool = False, **kwargs)#

Make sure python3 is installed on the remote nodes, and optionally make it the default.

It inherits the arguments of enoslib.api.actions.

Gather facts about hosts.

This function can be used to check/save the information of the infrastructure where the experiment ran. It’ll give the information gathered by Ansible (see https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html )

Parameters:

pattern_hosts (str) – pattern to describe ansible hosts to target. see https://docs.ansible.com/ansible/latest/intro_patterns.html
gather_subset (str) – if supplied, restrict the additional facts collected to the given subset. https://docs.ansible.com/ansible/latest/modules/setup_module.html
inventory_path (str) – inventory to use
roles (dict) – the roles to use (replacement for inventory_path).
extra_vars (dict) – extra_vars to use
on_error_continue (bool) – Don’t throw any exception in case a host is unreachable or the playbooks run with errors

Raises:

enoslib.errors.EnosFailedHostsError – if a task returns an error on a host and on_error_continue==False
enoslib.errors.EnosUnreachableHostsError – if a host is unreachable (through ssh) and on_error_continue==False

Returns:

Dict combining the ansible facts of ok and failed hosts and every result of tasks executed.

Example:

# Inventory
[control1]
enos-0
[control2]
enos-1

# Python
result = gather_facts(roles=roles)

# Result
{
    'failed': {},
    'ok':
    {
      'enos-0':
      {
        'ansible_product_serial': 'NA',
        'ansible_form_factor': 'Other',
        'ansible_user_gecos': 'root',
        ...
      },
      'enos-1':
      {...}
    'results': [...]
}

enoslib.api.generate_inventory(roles: Roles, networks: Networks, inventory_path: str, check_networks: bool = False)#

Generate an inventory file in the ini format.

The inventory is generated using the roles in the ini format. If check_network == True, the function will try to discover which networks interfaces are available and map them to one network of the networks parameters. Note that this auto-discovery feature requires the servers to have their IP set.

Parameters:

roles – role->hosts mapping as returned by enoslib.infra.provider.Provider.init()
networks – role->networks mapping as returned by enoslib.infra.provider.Provider.init()
inventory_path – path to the inventory to generate
check_networks – True to sync the hosts before dumping the inventory.

enoslib.api.get_hosts(roles: Roles, pattern_hosts: str = 'all') → List[Host]#

Get all the hosts matching the pattern.

Parameters:

roles – the roles as returned by enoslib.infra.provider.Provider.init()
pattern_hosts – pattern to describe ansible hosts to target. see https://docs.ansible.com/ansible/latest/intro_patterns.html

Returns:

The list of hosts matching the pattern

enoslib.api.run(cmd: str, roles: Roles | Iterable[Host] | Host, **kwargs) → Results#

Run command on some hosts

Parameters:

cmd – the command to run. This accepts Ansible templates
roles – host on which to run the command
kwargs – keyword argument of py:func:~enoslib.api.run_command

Returns:

Dict combining the stdout and stderr of ok and failed hosts and every result of tasks executed (this may include the fact gathering tasks)

Run Ansible.

Parameters:

roles
playbooks (list) – list of paths to the playbooks to run
inventory_path (str) – path to the hosts file (inventory)
extra_vars (dict) – extra vars to pass
tags (list) – list of tags to run
on_error_continue (bool) – Don’t throw any exception in case a host is unreachable or the playbooks run with errors
basedir – Ansible basedir

Raises:

enoslib.errors.EnosFailedHostsError – if a task returns an error on a host and on_error_continue==False
enoslib.errors.EnosUnreachableHostsError – if a host is unreachable (through ssh) and on_error_continue==False

enoslib.api.run_command(command: str, *, pattern_hosts: str = 'all', inventory_path: str | None = None, roles: Roles | Iterable[Host] | Host | None = None, gather_facts: bool = False, extra_vars: MutableMapping | None = None, on_error_continue: bool = False, run_as: str | None = None, background: bool = False, task_name: str | None = None, raw: bool = False, ns: str | None = None, cgroup: str | None = None, cgroup_prefix='/sys/fs/cgroup', **kwargs: Any) → Results#

Run a shell command on some remote hosts.

Parameters:

command (str) – the command to run
pattern_hosts (str) – pattern to describe ansible hosts to target. see https://docs.ansible.com/ansible/latest/intro_patterns.html
inventory_path (str) – inventory to use
roles (dict) – the roles to use (replacement for inventory_path).
extra_vars (dict) – extra_vars to use
gather_facts – True wheter facts should be gathered prior to the execution. Might be useful if Ansible variables are used.
on_error_continue (bool) – Don’t throw any exception in case a host is unreachable or the playbooks run with errors
run_as (str) – run the command as this user. This is equivalent to passing become=yes and become_user=user but become_method can be passed to modify the privileged escalation method. (default to sudo).
background (bool) – run the remote command in the background (detached mode) This is equivalent to passing async=one_year, poll=0
task_name – name of the command to display, can be used for further filtering once the results is retrieved.
raw – Whether to use a raw connection (no python requires at the destination)
ns – start the command in a pid namespace with that identifier
cgroup – start the command in the given cgroup (v2)
cgroup_prefix – where to find the cgroup filesystem (v2)
kwargs – keywords argument to pass to the shell module or as top level args.

Raises:

enoslib.errors.EnosFailedHostsError – if a task returns an error on a host and on_error_continue==False
enoslib.errors.EnosUnreachableHostsError – if a host is unreachable (through ssh) and on_error_continue==False

Returns:

Dict combining the stdout and stderr of ok and failed hosts and every result of tasks executed (this may include the fact gathering tasks)

Example:

# Inventory
[control1]
enos-0
[control2]
enos-1

# Python
result = run_command("date", inventory)

# Result
{
    'failed': {},
    'ok':
    {
        u'enos-0':
        {
            'stderr': u'',
            'stdout': u'Tue Oct 31 04:53:04 GMT 2017'
        },
        u'enos-1':
        {
            'stderr': u'',
            'stdout': u'Tue Oct 31 04:53:05 GMT 2017'}
        },
    'results': [...]
}

If facts are gathered it is possible to use ansible templating

result = run_command("control*", "ping -c 1
{{hostvars['enos-1']['ansible_' + n1].ipv4.address}}", inventory)

Command can be run asynchronously using the corresponding Ansible options (see https://docs.ansible.com/ansible/latest/user_guide/playbooks_async.html)

result = run_command("date", roles=roles, async=20, poll=0)

Note that the actual result isn’t available in the result file but will be available through a file specified in the result object.

Run a play.

Parameters:

play_source (dict) – ansible task
inventory_path (str) – inventory to use
hosts – host like datastructure used as a drop in replacement for an inventory.
extra_vars (dict) – extra_vars to use
on_error_continue (bool) – Don’t throw any exception in case a host is unreachable or the playbooks run with errors

Raises:

enoslib.errors.EnosFailedHostsError – if a task returns an error on a host and on_error_continue==False
enoslib.errors.EnosUnreachableHostsError – if a host is unreachable (through ssh) and on_error_continue==False

Returns:

List of all the results

enoslib.api.sync_info(roles: Roles, networks: Networks, inplace: bool = False, **kwargs) → Roles#

enoslib.api.sync_info(roles: Host, networks: Networks, inplace: bool = False, **kwargs) → Host

enoslib.api.sync_info(roles: Iterable[Host], networks: Networks, inplace: bool = False, **kwargs) → Iterable[Host]

Sync each host network information with their actual configuration

If the command is successful some host attributes will be populated. This allows to resync the enoslib Host representation with the remote configuration.

This method is generic: should work for any provider and supports IPv4 and IPv6 addresses.

Parameters:

roles (dict) – role->hosts mapping as returned by enoslib.infra.provider.Provider.init()
networks (list) – network list as returned by enoslib.infra.provider.Provider.init()
inplace – bool, default False If False, return a copy of roles. Otherwise, do operation inplace.
kwargs – keyword arguments passed to enoslib.api.run_ansible()

Returns:

RolesLike of the same type as passed. With updated information.

enoslib.api.wait_for(roles: Roles | Iterable[Host] | Host, retries: int = 100, interval: int = 30, **kwargs)#

Wait for all the machines to be ready to run some commands.

Let Ansible initiates a communication and retries if needed. Communication backend depends on the connection plugin used. This is most likely SSH but alternative backend can be used (see connection plugins)

Parameters:

roles – Roles to wait for
retries (int) – Number of time we’ll be retrying a connection
interval (int) – Interval to wait in seconds between two retries
kwargs – keyword arguments passed to enoslib.api.run_ansible()