Using the Tasks API

In this tutorial, you will learn how to organize your code into tasks and integrate it with a command line interface.

For the sake of illustration, let’s consider the following scenario:

  • Get machines from vagrant
  • Apply some network constraints between your machines
  • Validate those network constraints

Installation

$ pip install enoslib

Note

It’s a good practice to use a virtualenv or python version manager like pyenv.

Using the API

The following enos.py implements the desired workflow.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
import logging

from enoslib.api import discover_networks
from enoslib.infra.enos_vagrant.provider import Enos_vagrant
from enoslib.infra.enos_vagrant.configuration import Configuration
from enoslib.service import Netem


logging.basicConfig(level=logging.INFO)

provider_conf = {
    "backend": "libvirt",
    "resources": {
        "machines": [{
            "roles": ["control"],
            "flavour": "tiny",
            "number": 1,
        }, {
            "roles": ["compute"],
            "flavour": "tiny",
            "number": 1,
        }],
        "networks": [{"cidr": "192.168.40.0/24", "roles": ["mynetwork"]}]
    }
}

tc = {
    "enable": True,
    "default_delay": "20ms",
    "default_rate": "1gbit",
}


# claim the resources
conf = Configuration.from_dictionnary(provider_conf)

provider = Enos_vagrant(conf)
roles, networks = provider.init()

roles = discover_networks(roles, networks)

netem = Netem(tc, roles=roles)
# apply network constraints
netem.deploy()

# validate network constraints
netem.validate()

# reset network constraints
netem.destroy()

# validate network constraints and saving in an alternative
netem.validate(output_dir="after_reset")
  • Lines 5-18 describe the wanted resources. Here we want two machines with roles control and compute respectively. These two nodes will have one network card configured using the same network whose role is n1.

    Note

    Machine roles and network roles are transparent to the EnOSlib. The semantic is left to the application using it.

  • Lines 19-23 describe some network constraints. Those constraints will be set between the nodes of the two groups control and compute on the network n1.

  • Lines 27-34 enforce the wanted workflow.

    Note

    Under the hoods, EnOSlib leverages Ansible for many routine tasks and thus an inventory must be generated. This is exactly the purpose of enoslib.api.generate_inventory() function. When check_networks is set, EnOSlib will auto-discover the mapping between the network roles and the available network interfaces. This is convenient when it comes to deal with non uniform (or non deterministic) network cards naming.

  • You can launch the script using :

    $ python enos.py
    
  • The content of the generated inventory should looks like the following:

    [control]
    enos-0 ansible_host=192.168.121.70 ansible_ssh_user=root ansible_port=22 ansible_ssh_private_key_file=/home/msimonin/workspace/repos/enoslib/docs/tutorials/using-tasks/.vagrant/machines/enos-0/libvirt/private_key ansible_ssh_common_args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null' n3=eth1 enos_devices="['eth1']"
    [compute]
    enos-1 ansible_host=192.168.121.170 ansible_ssh_user=root ansible_port=22 ansible_ssh_private_key_file=/home/msimonin/workspace/repos/enoslib/docs/tutorials/using-tasks/.vagrant/machines/enos-1/libvirt/private_key ansible_ssh_common_args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null' n3=eth1 enos_devices="['eth1']"
    
  • You can check the generated reports by enoslib.api.validate_network() in _tmp_enos_.

    192.168.40.245 : 0.02 0.05 0.08 0.03 0.08 0.09 0.03 0.08 0.08 0.04
    192.168.40.244 : 40.75 40.65 40.78 40.69 41.20 41.45 41.59 41.61 41.64 40.36
    
           2 targets
           2 alive
           0 unreachable
           0 unknown addresses
    
           0 timeouts (waiting for response)
          20 ICMP Echos sent
          20 ICMP Echo Replies received
           0 other ICMP received
    
     0.02 ms (min round trip time)
     20.5 ms (avg round trip time)
     41.6 ms (max round trip time)
            9.075 sec (elapsed real time)
    
    

Using tasks

import os

from enoslib.api import discover_networks
from enoslib.infra.enos_vagrant.provider import Enos_vagrant
from enoslib.infra.enos_vagrant.configuration import Configuration
from enoslib.service import Netem
from enoslib.task import enostask

import logging

logging.basicConfig(level=logging.INFO)

provider_conf = {
    "backend": "libvirt",
    "resources": {
        "machines": [{
            "roles": ["control"],
            "flavour": "tiny",
            "number": 1,
        }, {
            "roles": ["compute"],
            "flavour": "tiny",
            "number": 1,
        }],
        "networks": [{"cidr": "192.168.40.0/24", "roles": ["mynetwork"]}]
    }
}

tc = {
    "enable": True,
    "default_delay": "20ms",
    "default_rate": "1gbit",
}


@enostask(new=True)
def up(force=True, env=None, **kwargs):
    "Starts a new experiment"
    conf = Configuration.from_dictionnary(provider_conf)
    provider = Enos_vagrant(conf)
    roles, networks = provider.init()
    roles = discover_networks(roles, networks)
    env["roles"] = roles
    env["networks"] = networks


@enostask()
def emulate(env=None, **kwargs):
    roles = env["roles"]
    netem = Netem(tc, roles=roles)
    netem.deploy()


@enostask()
def validate(env=None, **kwargs):
    roles = env["roles"]
    netem = Netem(tc, roles=roles)
    netem.validate()

up()
emulate()
validate()
  • Using Tasks is a neat way to organize your program into a workflow.

  • The environment (env variable in the above) is a way to (1) store information on the current execution and (2) pass information from one task to another. It is automatically restored at the beginning of a task and saved at the end.

  • You can launch the script using :

    $ python enos.py
    

Integrating with a command line parser

Let’s integrate our tasks with a command line parser. Here we choose click. First ensure that it is installed:

$ pip install click
  • Change the content of enos.py to the following:

    import logging
    import os
    
    from enoslib.api import discover_networks
    from enoslib.infra.enos_vagrant.provider import Enos_vagrant
    from enoslib.infra.enos_vagrant.configuration import Configuration
    from enoslib.service import Netem
    from enoslib.task import enostask
    
    
    logging.basicConfig(level=logging.INFO)
    
    provider_conf = {
        "backend": "libvirt",
        "resources": {
            "machines": [{
                "roles": ["control"],
                "flavour": "tiny",
                "number": 1,
            }, {
                "roles": ["compute"],
                "flavour": "tiny",
                "number": 1,
            }],
            "networks": [{"cidr": "192.168.40.0/24", "roles": ["mynetwork"]}]
        }
    }
    
    tc = {
        "enable": True,
        "default_delay": "20ms",
        "default_rate": "1gbit",
    }
    
    
    import click
    @click.group()
    def cli():
        pass
    
    
    @cli.command()
    @click.option("--force",is_flag=True, help="vagrant destroy and up")
    @enostask(new=True)
    def up(force, env=None, **kwargs):
        """Starts a new experiment using vagrant"""
        conf = Configuration.from_dictionnary(provider_conf)
        provider = Enos_vagrant(conf)
        roles, networks = provider.init(force_deploy=force)
        roles = discover_networks(roles, networks)
        env["roles"] = roles
        env["networks"] = networks
    
    
    @cli.command()
    @enostask()
    def emulate(env=None, **kwargs):
        """Emulates the network."""
        roles = env["roles"]
        netem = Netem(tc, roles=roles)
        netem.deploy()
    
    
    @cli.command()
    @enostask()
    def validate(env=None, **kwargs):
        """Validates the network constraints."""
        roles = env["roles"]
        netem = Netem(tc, roles=roles)
        netem.validate()
    
    
    
    if __name__ == '__main__':
        cli()
    
  • For the sake of illustration, we added a flag (--force) to the command line which allows to force the recreation of the virtual machines (see enoslib.infra.provider.Provider.init()). Since every provider supports this flag we pass its value to the ìnit method.

  • You will now have access to the command line interface :

    $ python enos.py --help
    Usage: enos.py [OPTIONS] COMMAND [ARGS]...
    
    Options:
      --help  Show this message and exit.
    
    Commands:
      emulate   Emulates the network.
      up        Starts a new experiment using vagrant
      validate  Validates the network constraints.