Provider::VMonG5K

This tutorial leverages the VmonG5k provider: a provider that provisions virtual machines for you on Grid’5000.

Hint

For a complete schema reference see VMonG5k Schema

Installation

On Grid’5000, you can go with a virtualenv :

$ virtualenv -p python3 venv
$ source venv/bin/activate
$ pip install -U pip

$ pip install enoslib

Configuration

Since python-grid5000 is used behind the scene, the configuration is read from a configuration file located in the home directory. It can be created with the following:

echo '
username: MYLOGIN
password: MYPASSWORD
' > ~/.python-grid5000.yaml

chmod 600 ~/.python-grid5000.yaml

With the above you can access the Grid’5000 API from you local machine aswell.

External access

If you want to control you experiment from the outside of Grid’5000 (e.g from your local machine) you can refer to the following. You can jump this section if you work from inside Grid’5000.

SSH external access

  • Solution 1: use the Grid’5000 VPN
  • Solution 2: configure you ~/.ssh/config properly:
Host *.grid5000.fr
ProxyCommand ssh -A <login>@194.254.60.33 -W "$(basename %h):%p"
User <login>
ForwardAgent yes

Accessing HTTP services inside Grid’5000

If you want to control you experiment from the outside of Grid’5000 (e.g from your local machine). For instance the Distem provider is starting a web server to handle the client requests. In order to access it propertly externally you drom your local machine can either

  • Solution 1 (general): use the Grid’5000 VPN

  • Solution 2 (HTTP traffic only): create a socks tunnel from your local machine to Grid’5000
    # on one shell
    ssh -ND 2100 access.grid5000.fr
    
    # on another shell
    export https_proxy="socks5h://localhost:2100"
    export http_proxy="socks5h://localhost:2100"
    
    # Note that browsers can work with proxy socks
    chromium-browser --proxy-server="socks5://127.0.0.1:2100" &
    
  • Solution 3 (ad’hoc): create a forwarding port tunnel

    # on one shell
    ssh -Nl 3000:paravance-42.rennes.grid5000.fr:3000 access.grid5000.fr
    
    # Now all traffic that goes on localhost:3000 is forwarded to paravance-42.rennes.grid5000.fr:3000
    

To accesss your virtual machines from your local machine, see below.

Basic example

We’ll imagine a system that requires 100 compute machines and 3 controller machines. We express this using the ~VmonG5K~ provider:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
from enoslib.api import discover_networks
from enoslib.infra.enos_vmong5k.provider import VMonG5k
from enoslib.infra.enos_vmong5k.configuration import Configuration

import logging
import os

logging.basicConfig(level=logging.DEBUG)

# claim the resources
conf = (
    Configuration
    .from_settings(job_name="tuto-vmong5k")
    .add_machine(
        roles=["compute"],
        cluster="grisou",
        number=1,
        flavour="tiny"
    )
    .add_machine(
        roles=["controller"],
        cluster="grisou",
        number=3,
        flavour="tiny"
    )
    .finalize()
)


provider = VMonG5k(conf)

roles, networks = provider.init()
print(roles)
print(networks)
  • You can launch the script using :

    $ python tuto_vmg5k.py
    
  • The raw data structures of EnOSlib will be displayed and you should be able to connect to any machine using SSH and the root account.

Notes

  • The VmonG5K provider internally uses the G5k provider. In particular it sets the job_type to allow_classic_ssh and claim an extra slash_22 subnet.
  • SSH access will be granted to the VMs using the ~/.ssh/id_rsa | ~/.ssh/id_rsa.pub keypair. So these files must be present in your home directory.
  • The working_dir setting controls where the temporary files and virtual images disks will be stored. The default is to store everything in the temp folder of the physical nodes.
  • You might be interested in adding wait_ssh(roles) (from enoslib.api) just after init() to make sure SSH is up and running on all VMs. Otherwise you might get an unreachable error from SSH.

Warning

The working_dir and all its content is deleted by the provider when calling destroy.

Changing resource size of virtual machines

As for the CPU and memory resources, you can simply change the name of the flavour (available flavours are listed here), or create your own flavour with flavour_desc.

[...]
.add_machine(
    [...],
    flavour_desc={"core": 1, "mem": "512"}
)

Notes on the disks of Virtual Machines

  • Adding a new disk: Using the disk attribute of flavour_desc will create a new disk and make it available to the VM. For instance to get an extra disk of 10GB you can use this python configuration parameter:

    [...]
    .add_machine(
        [...],
        flavour_desc={"core": 1, "mem": 512, "disk": 10}
    )
    

Note that with the above configuration an extra disk of 10GB will be provisioned and available to the Virtual Machine. In the current implementation, the disk is neither formatted nor mounted in the Virtual Machine OS.

  • Make an external (not managed by EnOSlib) disk available to the Virtual Machine A typical use case is to use an hardware disk from the host machine. In this situation, use the extra_devices parameter of the configuration. It corresponds to the XML string of Libvirt.

    [...]
    .add_machine(
        [...],
        extra_devices = """
        <disk type='block' device='disk'>
        <driver name='qemu' type='raw'/>
        <source dev='/dev/disk/by-path/pci-0000:82:00.0-sas-phy1-lun-0'/>
        <target dev='vde' bus='virtio'/>
        </disk>
        """
    
  • Resize the root filesystem To do so, you will need to get the qcow2 file, put it in your public folder, and resize it. Location of the file is shown here.

    cp /grid5000/virt-images/debian10-x64-nfs.qcow2 $HOME/public/original.qcow2
    cd $HOME/public
    qemu-img info original.qcow2  # check the size (should be 10GB)
    qemu-img resize original.qcow2 +30G
    # at this stage, image is resized at 40GB but not the partition
    virt-resize --expand /dev/sda1 original.qcow2 my-image.qcow2
    rm original.qcow2
    # now you can check the size of each partition (/dev/sda1 should be almost 40GB)
    virt-filesystems –long -h –all -a my-image.qcow2
    

    Finally, you need to tell EnosLib to use this file with:

    Configuration.from_settings(...
                                image="/home/<username>/public/my-image.qcow2",
                                ...
                                )
    

EnOSlib primer using VMonG5k

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
from enoslib.api import discover_networks, play_on
from enoslib.infra.enos_vmong5k.provider import VMonG5k
from enoslib.infra.enos_vmong5k.configuration import Configuration

import logging


logging.basicConfig(level=logging.DEBUG)

conf = (
    Configuration
    .from_settings(
        job_name="flent_on",
        gateway=True
    )
    .add_machine(
        roles=["server"],
        cluster="paravance",
        number=1
    )
    .add_machine(
        roles=["client"],
        cluster="paravance",
        number=1
    )
    .finalize()
)

provider = VMonG5k(conf)

roles, networks = provider.init()
roles = discover_networks(roles, networks)
with play_on(roles=roles) as p:
    # flent requires python3, so we default python to python3
    p.shell("update-alternatives --install /usr/bin/python python /usr/bin/python3 1")
    p.apt_repository(repo="deb http://deb.debian.org/debian stretch main contrib non-free",
                     state="present")
    p.apt(name=["flent", "netperf", "python3-setuptools", "python3-matplotlib"],
          state="present")

with play_on(pattern_hosts="server", roles=roles) as p:
    p.shell("nohup netperf &")

with play_on(pattern_hosts="client", roles=roles) as p:
    p.shell("flent rrul -p all_scaled "
            + "-l 60 "
            + "-H {{ hostvars[groups['server'][0]].ansible_default_ipv4.address }} "
            + "-t 'bufferbloat test' "
            + "-o result.png")
    p.fetch(src="result.png",
            dest="result")

SSH external access to the virtual machines

This is mandatory if you deployed from your local machine.

  • Solution 1: use the Grid’5000 VPN

  • Solution 2: Add the following in your configuration force Ansible to jump through a gateway (access.grid5000.fr):

    :: code-block:: python

    Configuration.from_settings(…

    gateway=True …

    )

Controlling the virtual machines placement

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
from enoslib.api import discover_networks
import enoslib.infra.enos_g5k.configuration as g5kconf
from enoslib.infra.enos_g5k.provider import G5k
import enoslib.infra.enos_vmong5k.configuration as vmconf
from enoslib.infra.enos_vmong5k.provider import start_virtualmachines

import logging

logging.basicConfig(level=logging.INFO)

CLUSTER = "parasilo"
SITE = "rennes"


prod_network = g5kconf.NetworkConfiguration(
    id="n1",
    type="prod",
    roles=["my_network"],
    site=SITE)
conf = (
    g5kconf.Configuration
    .from_settings(
        job_type="allow_classic_ssh",
        job_name="placement"
    )
    .add_network_conf(prod_network)
    .add_network(
        id="not_linked_to_any_machine",
        type="slash_22",
        roles=["my_subnet"],
        site=SITE
    )
    .add_machine(
        roles=["role1"],
        cluster=CLUSTER,
        nodes=1,
        primary_network=prod_network
    )
    .add_machine(
        roles=["role2"],
        cluster=CLUSTER,
        nodes=1,
        primary_network=prod_network
    )
    .finalize()
 )

provider = G5k(conf)
roles, networks = provider.init()
roles = discover_networks(roles, networks)

# Retrieving subnet
subnet = [n for n in networks if "my_subnet" in n["roles"]]
logging.info(subnet)

# We describe the VMs types and placement in the following
virt_conf = (
    vmconf.Configuration
    .from_settings(image="/grid5000/virt-images/debian9-x64-std.qcow2")
    # Starts some vms on a single role
    # Here that means start the VMs on a single machine
    .add_machine(
        roles=["vms"],
        number=16,
        undercloud=roles["role1"]
    )
    .finalize()
)

# Start them
vmroles, networks = start_virtualmachines(virt_conf, subnet)
print(vmroles)
print(networks)