Welcome to EnOSlib’s documentation!

EnosLib

Build Status License Pypi

Join us on gitter : Join the chat at https://gitter.im/BeyondTheClouds/enoslib

EnOSlib helps you to drive your experiments on various infrastructures. To do so, most of your experiment logic is made reusable by EnOSlib (getting resources, launching remote actions, deploying side analysis stacks …)

More pragmatically, with EnOSlib, you can iterate on your application deployment and experimental workflow locally before moving to a large testbed like Grid’5000, or Chameleon. It saves time and energy.

EnOSlib is designed for experimentation purpose: benchmark in a controlled environment, academic validation …

EnOSLib is developed in the context of the Discovery initiative

EnOSlib primer

Let’s consider a user called Alice or Bob.

{{ user }} would like to start a network benchmark between the nodes of an infrastructure. {{ user }} chooses to go with flent and thus writes the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
from enoslib.api import discover_networks, play_on
from enoslib.infra.enos_vagrant.provider import Enos_vagrant
from enoslib.infra.enos_vagrant.configuration import Configuration

import logging


logging.basicConfig(level=logging.DEBUG)

# The conf let us define the resources wanted.
# This is provider specific
conf = Configuration.from_settings(backend="libvirt",
                                   box="generic/debian9")\
                    .add_machine(roles=["server"],
                                 flavour="tiny",
                                 number=1)\
                    .add_machine(roles=["client"],
                                 flavour="tiny",
                                 number=1)\
                    .add_network(roles=["mynetwork"],
                                 cidr="192.168.42.0/24")\
                    .finalize()

provider = Enos_vagrant(conf)

# The code below is intended to be provider agnostic

# Start the resources
roles, networks = provider.init()

# Add some specific knowledge to the returned roles (e.g on the server the ip
# for mynetwork is 192.168.42.254)
discover_networks(roles, networks)

# Experimentation logic starts here
with play_on(roles=roles) as p:
    # flent requires python3, so we default python to python3
    p.shell("update-alternatives --install /usr/bin/python python /usr/bin/python3 1")
    p.apt_repository(repo="deb http://deb.debian.org/debian stretch main contrib non-free",
                     state="present")
    p.apt(name=["flent", "netperf", "python3-setuptools"],
          state="present")

with play_on(pattern_hosts="server", roles=roles) as p:
    p.shell("nohup netperf &")

with play_on(pattern_hosts="client", roles=roles) as p:
    p.shell("flent rrul -p all_scaled "
            + "-l 60 "
            + "-H {{ hostvars[groups['server'][0]].inventory_hostname }} "
            + "-t 'bufferbloat test' "
            + "-o result.png")
    p.fetch(src="result.png",
            dest="result")

This starts {{ user }}’s experiment on the local machine using Vagrant (with libvirt). Note that a {{ user }}’s friend will be able to run the same using backend="virtualbox" if VirtualBox is installed. Now looking at the result directory created at the end of the execution, {{ user }} finds:

_images/result.png

{{ user }} launches several times the script, getting new results. Subsequent runs are faster because the machines are already up and everything is idempotent.

{{ user }} now would like to go in a real testbed (e.g Grid’5000). Good news ! {{ user }} only have to adapt the configuration phase and the provider used in the script. The experimentation logic can remain the same. Thus, one can write the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
from enoslib.api import discover_networks, play_on
from enoslib.infra.enos_g5k.provider import G5k
from enoslib.infra.enos_g5k.configuration import (Configuration,
                                                  NetworkConfiguration)

import logging


logging.basicConfig(level=logging.DEBUG)

network = NetworkConfiguration(id="n1",
                               type="kavlan",
                               roles=["mynetwork"],
                               site="rennes")
conf = Configuration.from_settings(job_name="flent_on",
                                  env_name="debian9-x64-std")\
                    .add_network_conf(network)\
                    .add_machine(roles=["server"],
                                 cluster="parapluie",
                                 nodes=1,
                                 primary_network=network)\
                    .add_machine(roles=["client"],
                                 cluster="parapluie",
                                 nodes=1,
                                 primary_network=network)\
                    .finalize()

provider = G5k(conf)
roles, networks = provider.init()
discover_networks(roles, networks)
with play_on(roles=roles) as p:
    # flent requires python3, so we default python to python3
    p.shell("update-alternatives --install /usr/bin/python python /usr/bin/python3 1")
    p.apt_repository(repo="deb http://deb.debian.org/debian stretch main contrib non-free",
                     state="present")
    p.apt(name=["flent", "netperf", "python3-setuptools"],
          state="present")

with play_on(pattern_hosts="server", roles=roles) as p:
    p.shell("nohup netperf &")

with play_on(pattern_hosts="client", roles=roles) as p:
    p.shell("flent rrul -p all_scaled "
            + "-l 60 "
            + "-H {{ hostvars[groups['server'][0]].inventory_hostname }} "
            + "-t 'bufferbloat test' "
            + "-o result.png")
    p.fetch(src="result.png",
            dest="result")
_images/result_g5k.png

Now where {{ user }} can go from here depends on the needs:

  • Moving to another provider is possible. {{ user }} only needs to learn about the specific object for this configuration – e.g see [1].
  • Refactoring the code, providing a command line interface could be also nice. After all its just python code – see [2]. Or moving the deployment code (because it becomes large) into separated Ansible files – see [3].
  • Applying specific network constraints between the nodes of the reservation is also possible. {{ user }}’ll need to learn more about how enforcing the constraints – see [4].
  • Deploying a monitoring stack can be convenient to track some system metrics. EnOSlib offers some services that can be used easily – see [5] or [6].
  • Contributing to this project would be wonderful. {{ user }} doesn’t need to do much, improving the documentation would be very helpful – see [5].