Welcome to EnOSlib’s documentation!

EnosLib

Build Status License Pypi

EnOSlib is a library to help you with your experiments. The main parts of your experiment logic is made reusable by the following EnOSlib building blocks:

  • Reusable infrastructure configuration: The provider abstraction allows you to run your experiment on different environments (locally with Vagrant, Grid’5000, Chameleon and more)
  • Reusable software provisioning: In order to configure your nodes, EnOSlib exposes different APIs with different level of expressivity
  • Reusable experiment facilities: Tasks help you to organize your experimentation workflow.

EnOSlib is designed for experimentation purpose: benchmark in a controlled environment, academic validation …

EnOSLib has been initially developed in the context of the Discovery initiative

EnOSlib primer

Let’s consider a user called Alice or Bob.

{{ user }} would like to start a network benchmark between the nodes of an infrastructure. {{ user }} chooses to go with flent and thus writes the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
from enoslib.api import discover_networks, play_on
from enoslib.infra.enos_vagrant.provider import Enos_vagrant
from enoslib.infra.enos_vagrant.configuration import Configuration

import logging


logging.basicConfig(level=logging.DEBUG)

# The conf let us define the resources wanted.
# This is provider specific
conf = (
    Configuration
    .from_settings(backend="libvirt")
    .add_machine(
        roles=["server"],
        flavour="tiny",
        number=1
    )
    .add_machine(
        roles=["client"],
        flavour="tiny",
        number=1
    )
    .add_network(
        roles=["mynetwork"],
        cidr="192.168.42.0/24")
    .finalize()
)

provider = Enos_vagrant(conf)

# The code below is intended to be provider agnostic

# Start the resources
roles, networks = provider.init()

# Add some specific knowledge to the returned roles (e.g on the server the ip
# for mynetwork is 192.168.42.254)
roles = discover_networks(roles, networks)

# Experimentation logic starts here
with play_on(roles=roles) as p:
    # flent requires python3, so we default python to python3
    p.shell("update-alternatives --install /usr/bin/python python /usr/bin/python3 1")
    p.apt_repository(repo="deb http://deb.debian.org/debian stretch main contrib non-free",
                     state="present")
    p.apt(name=["flent", "netperf", "python3-setuptools", "python3-matplotlib"],
          state="present")

with play_on(pattern_hosts="server", roles=roles) as p:
    p.shell("nohup netperf &")

with play_on(pattern_hosts="client", roles=roles) as p:
    p.shell("flent rrul -p all_scaled "
            + "-l 60 "
            + "-H {{ hostvars[groups['server'][0]].ansible_default_ipv4.address }} "
            + "-t 'bufferbloat test' "
            + "-o result.png")
    p.fetch(src="result.png",
            dest="result")

This starts {{ user }}’s experiment on the local machine using Vagrant (with libvirt). Note that a {{ user }}’s friend will be able to run the same using backend="virtualbox" if VirtualBox is installed. Now looking at the result directory created at the end of the execution, {{ user }} finds:

_images/result.png

{{ user }} launches several times the script, getting new results. Subsequent runs are faster because the machines are already up and everything is idempotent.

{{ user }} now would like to go in a real testbed (e.g Grid’5000). Good news ! {{ user }} only have to adapt the configuration phase and the provider used in the script. The experimentation logic can remain the same. Thus, one can write the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
from enoslib.api import discover_networks, play_on
from enoslib.infra.enos_g5k.provider import G5k
from enoslib.infra.enos_g5k.configuration import (Configuration,
                                                  NetworkConfiguration)

import logging


logging.basicConfig(level=logging.DEBUG)

network = NetworkConfiguration(
    id="n1",
    type="kavlan",
    roles=["mynetwork"],
    site="rennes"
)

conf = (
    Configuration
    .from_settings(job_name="flent_on")
    .add_network_conf(network)
    .add_machine(
        roles=["server"],
        cluster="parapluie",
        nodes=1,
        primary_network=network
    )
    .add_machine(
        roles=["client"],
        cluster="parapluie",
        nodes=1,
        primary_network=network
    )
    .finalize()
)

provider = G5k(conf)
roles, networks = provider.init()
roles = discover_networks(roles, networks)
with play_on(roles=roles) as p:
    # flent requires python3, so we default python to python3
    p.shell("update-alternatives --install /usr/bin/python python /usr/bin/python3 1")
    p.apt_repository(repo="deb http://deb.debian.org/debian stretch main contrib non-free",
                     state="present")
    p.apt(name=["flent", "netperf", "python3-setuptools", "python3-matplotlib"],
          state="present")

with play_on(pattern_hosts="server", roles=roles) as p:
    p.shell("nohup netperf &")

with play_on(pattern_hosts="client", roles=roles) as p:
    p.shell("flent rrul -p all_scaled "
            + "-l 60 "
            + "-H {{ hostvars[groups['server'][0]].ansible_default_ipv4.address }} "
            + "-t 'bufferbloat test' "
            + "-o result.png")
    p.fetch(src="result.png",
            dest="result")
_images/result_g5k.png

Now where {{ user }} can go from here depends on the needs:

  • Moving to another provider is possible. {{ user }} only needs to learn about the specific object for this configuration – e.g see [1a], [1b].
  • Refactoring the code, providing a command line interface could be also nice. After all its just python code – see [2]. Or moving the deployment code (because it becomes large) into separated Ansible files – see [3].
  • Applying specific network constraints between the nodes of the reservation is also possible. {{ user }}’ll need to learn more about how enforcing the constraints – see [4].
  • Deploying a monitoring stack can be convenient to track some system metrics. EnOSlib offers some services that can be used easily – see [5] or [6].
  • Contributing to this project would be wonderful. {{ user }} doesn’t need to do much, improving the documentation would be very helpful – see [5].