Distem tutorials

This tutorial leverages the Distem provider: a provider that creates containers for you on Grid’5000.

Note

More details on : https://distem.gitlabpages.inria.fr/

Hint

For a complete schema reference see Distem Schema

Installation

To use Grid’5000 with Enoslib, you can go with a virtualenv :

$ virtualenv -p python3 venv
$ source venv/bin/activate
$ pip install -U pip

$ pip install enoslib

Configuration

Since python-grid5000 is used behind the scene, the configuration is read from a configuration file located in the home directory. It can be created with the following:

echo '
username: MYLOGIN
password: MYPASSWORD
' > ~/.python-grid5000.yaml

chmod 600 ~/.python-grid5000.yaml

The above configuration should work both from a Grid’5000 frontend machine and from your local machine as well.

External access (from your laptop)

If you are running your experiment from outside Grid’5000 (e.g. from your local machine), using a SSH jump host is required.

Enoslib (version 8.1.0 and above) will automatically setup such a SSH jump host connection through access.grid5000.fr when it detects you are working outside of the Grid’5000 network. See Global configuration if you need to configure this behaviour.

Hint

Using a SSH jump host does not provide the best performance when controlling a large number of nodes. This is because the number of simultaneous SSH connection is limited on the jump host. See Performance tuning for many tips and tricks to improve performance.

Basic example

We’ll imagine a system that requires 50 compute machines and 1 controller machines. We express this using the Distem provider:

Hint

• If you don’t have any image :

• $) wget ‘http://public.nancy.grid5000.fr/~msimonin/public/distem-stretch.tar.gz’ -P /home/public

from pathlib import Path

import enoslib as en

FORCE = False
CLUSTER = "parasilo"

en.init_logging()
en.check()

job_name = Path(__file__).name


# claim the resources
conf = (
    en.DistemConf.from_settings(
        job_name=job_name,
        force_deploy=FORCE,
        image="file:///home/msimonin/public/distem-stretch.tgz",
    )
    .add_machine(roles=["server"], cluster=CLUSTER, number=1, flavour="large")
    .add_machine(roles=["client"], cluster=CLUSTER, number=1, flavour="large")
)

provider = en.Distem(conf)

roles, networks = provider.init()

print(roles)
print(networks)
gateway = networks[0].gateway
print("Gateway : %s" % gateway)

roles = en.sync_info(roles, networks)

with en.actions(roles=roles, gather_facts=False) as p:
    # We first need internet connectivity
    # Netmask for a subnet in g5k is a /14 netmask
    p.shell("ifconfig if0 $(hostname -I) netmask 255.252.0.0")
    p.shell("route add default gw %s dev if0" % gateway)


# Experimentation logic starts here
with en.actions(roles=roles) as p:
    # flent requires python3, so we default python to python3
    p.apt_repository(
        repo="deb http://deb.debian.org/debian stretch main contrib non-free",
        state="present",
    )
    p.apt(name=["flent", "netperf", "python3-setuptools"], state="present")

with en.actions(pattern_hosts="server", roles=roles) as p:
    p.shell("nohup netperf &")

with en.actions(pattern_hosts="client", roles=roles) as p:
    p.shell(
        "flent rrul -p all_scaled "
        + "-l 60 "
        + "-H {{ hostvars[groups['server'][0]].inventory_hostname }} "
        + "-t 'bufferbloat test' "
        + "-o result.png"
    )
    p.fetch(src="result.png", dest="result")

Note

• You can customize your virtual environment as explained in : https://distem.gitlabpages.inria.fr/faq.html#how-to-customize-your-virtual-environment

• Alternatively a quick way to create a base virtual environment (from amerlin):

lxc-create -n myimg -t download --  --dist debian --release stretch --arch amd64
mount -o bind /dev /var/lib/lxc/myimg/rootfs/dev
chroot /var/lib/lxc/myimg/rootfs
rm /etc/resolv.conf
echo "nameserver 9.9.9.9" > /etc/resolv.conf
# distem requirements: sshd
apt install openssh-server
# enoslib requirements: python
apt install -y python3
update-alternatives --install /usr/bin/python python /usr/bin/python3 1
# your configuration goes here
exit
umount /var/lib/lxc/myimg/rootfs/dev
cd /var/lib/lxc/myimg/rootfs
tar -czvf ../distem-stretch.tgz .

EnOSlib bootsraps distem server and agents on your nodes and start the container for you. In particular:

• the distem coordinator runs on one of your node (randomly picked)

• one SSH keypair is generated for the inter-nodes connectivity and the vnodes external access

• EnOSlib SSH connection to the vnodes are tunneled through the coordinator node (make sure to do the same if you want to SSH manually)

• One single network (slash_22) is currently supported (https://gitlab.inria.fr/discovery/enoslib/issues/38)

• flavour configuration is ignored currently (https://gitlab.inria.fr/discovery/enoslib/issues/37)