Ansible Integration#

EnOSlib uses internally Ansible to perform some predefined tasks (e.g remote commands ). But in EnOSlib we prefer not leaving the Python world :) so we’ve exposed some convenient functions that use Ansible behind the scene like:

  • run a single command on a subset of the nodes and gather the outputs (and errors)

  • or run a set of actions (Ansible Modules) programmatically

Hint

The presented methods are provider agnostic.

Run a single command on nodes and gather results#

Let’s consider the following script :

 1import enoslib as en
 2
 3en.init_logging()
 4
 5provider_conf = {
 6    "backend": "libvirt",
 7    "resources": {
 8        "machines": [
 9            {
10                "roles": ["control1"],
11                "flavour": "tiny",
12                "number": 1,
13            },
14            {
15                "roles": ["control2"],
16                "flavour": "tiny",
17                "number": 1,
18            },
19        ],
20        "networks": [{"roles": ["rn1"], "cidr": "172.16.0.1/16"}],
21    },
22}
23
24conf = en.VagrantConf.from_dictionary(provider_conf)
25provider = en.Vagrant(conf)
26roles, networks = provider.init()
27roles = en.sync_info(roles, networks)
28
29
30result = en.run_command("date", roles=roles)
31print(result)
32
33# use a list of Hosts
34result = en.run_command("date", roles=roles["control1"])
35print(result)
36
37# use a single Hosts
38result = en.run_command("date", roles=roles["control1"][0])
39print(result)
40
41# filter hosts using a pattern
42result = en.run_command("date", pattern_hosts="control*", roles=roles)
43print(result)
44
45# async tasks / will run in detached mode
46result = en.run_command("date", roles=roles, background=True)
47print(result)
  • run_command() takes at least 2 parameters :

    • the actual command to run

    • the Roles (as returned by the provider init method), or an iterable of Host, or a single Host

  • A Results object is returned and allow for further filtering.

Run a set of actions on nodes#

Using python exclusively#

Let’s consider the following script:

 1import logging
 2
 3import enoslib as en
 4
 5en.init_logging(level=logging.INFO)
 6
 7# The conf let us define the resources wanted.
 8# This is provider specific
 9conf = (
10    en.VagrantConf()
11    .from_settings(backend="libvirt")
12    .add_machine(roles=["server"], flavour="tiny", number=1)
13    .add_machine(roles=["client"], flavour="tiny", number=1)
14    .add_network(roles=["mynetwork"], cidr="192.168.42.0/24")
15)
16
17provider = en.Vagrant(conf)
18
19# The code below is intended to be provider-agnostic
20
21# Start the resources
22roles, networks = provider.init()
23
24
25# Experimentation logic starts here
26with en.actions(roles=roles) as p:
27    p.apt_repository(
28        repo="deb http://deb.debian.org/debian stretch main contrib non-free",
29        state="present",
30    )
31    p.apt(
32        name=["flent", "netperf", "python3-setuptools", "python3-matplotlib"],
33        state="present",
34    )
35
36with en.actions(pattern_hosts="server", roles=roles) as p:
37    p.shell("nohup netperf &")
38
39with en.actions(pattern_hosts="client", roles=roles) as p:
40    # get the address of server
41    server_address = roles["server"][0].address
42    p.shell(
43        "flent rrul -p all_scaled "
44        + "-l 60 "
45        + f"-H { server_address } "
46        + "-t 'bufferbloat test' "
47        + "-o result.png"
48    )
49    p.fetch(src="result.png", dest="result")

flent_on.py

In this example each actions block run a playbook generated from the ansible module calls made. Any ansible module can be called here, the exact keyword arguments to pass depend on each module and you’ll need to refer to the ansible modules documentation (e.g https://docs.ansible.com/ansible/latest/modules/apt_module.html).

Re-using results from a previous task#

To access results from previous tasks, you need to start a new actions block. Here is an example with Garage:

 1import logging
 2from pathlib import Path
 3
 4import enoslib as en
 5
 6en.init_logging(level=logging.INFO)
 7en.check()
 8
 9GARAGE_URL = (
10    "https://garagehq.deuxfleurs.fr/_releases/v0.7.3/"
11    "x86_64-unknown-linux-musl/garage"
12)
13job_name = Path(__file__).name
14
15conf = (
16    en.G5kConf()
17    .from_settings(job_name=job_name, walltime="0:40:00")
18    .add_machine(roles=["garage"], cluster="econome", nodes=2)
19)
20
21provider = en.G5k(conf)
22
23roles, networks = provider.init()
24
25with en.actions(roles=roles["garage"], gather_facts=True) as p:
26    p.get_url(
27        task_name="Download garage",
28        url=GARAGE_URL,
29        dest="/tmp/garage",
30        mode="755",
31    )
32    p.template(
33        task_name="Create config",
34        src=str(Path(__file__).parent / "garage.toml.j2"),
35        dest="/tmp/garage.toml",
36    )
37    p.command(
38        task_name="Kill garage if already running",
39        cmd="killall garage",
40        ignore_errors=True,
41    )
42    p.command(
43        task_name="Run garage in the background",
44        cmd="/tmp/garage -c /tmp/garage.toml server",
45        background=True,
46    )
47    p.command(
48        task_name="Get node ID",
49        cmd="/tmp/garage -c /tmp/garage.toml node id -q",
50    )
51
52# Collect Garage nodes ID in a dictionary
53results = p.results
54nodes_id = {r.host: r.stdout for r in results.filter(task="Get node ID")}
55
56with en.actions(roles=roles["garage"], gather_facts=False) as p:
57    for remote_node, remote_id in nodes_id.items():
58        p.command(
59            task_name=f"Connect to remote node {remote_node}",
60            cmd=f"/tmp/garage -c /tmp/garage.toml node connect {remote_id}",
61        )
62
63
64# Release all Grid'5000 resources
65provider.destroy()

garage.py

garage.toml.j2

Using a yaml playbook#

In addition to run custom command on nodes, EnOSlib can trigger predefined playbook stored in your filesystem. This lets you use the native Ansible syntax to describe your tasks or launch an existing playbook.

Let’s consider the following script:

 1import enoslib as en
 2
 3en.init_logging()
 4
 5
 6provider_conf = {
 7    "resources": {
 8        "machines": [
 9            {
10                "roles": ["control1"],
11                "flavour": "tiny",
12                "number": 1,
13            },
14            {
15                "roles": ["control2"],
16                "flavour": "tiny",
17                "number": 1,
18            },
19        ],
20        "networks": [{"roles": ["rn1"], "cidr": "172.16.0.1/16"}],
21    }
22}
23
24conf = en.VagrantConf.from_dictionary(provider_conf)
25provider = en.Vagrant(conf)
26roles, networks = provider.init()
27
28result = en.run_ansible(["site.yml"], roles=roles)
29print(result)

run_ansible.py

And the corresponding minimal playbook site.yml:

1---
2- name: This is a play
3  hosts: control*
4  tasks:
5    - name: One task
6      debug:
7        msg: "I'm running on {{ inventory_hostname }}"

site.yml

What’s next ?#

If you aren’t familiar with Ansible, we encourage you to go through the Ansible documentation, write your own playbooks, or integrate one existing from ansible galaxy.