Provider::Grid’5000

This tutorial illustrates the use of EnOSlib to interact with Grid’5000. For a full description of the API and the available options, please refer to the API documentation of the Grid’5000 provider.

Hint

For a complete schema reference see G5k Schema

Hint

EnOSlib has also a dedicated tutorial at https://msimonin.gitlabpages.inria.fr/enoslib-tutorials/ It illustrates the use of EnOSlib in the Grid’5000 context.

Installation

On Grid’5000, you can go with a virtualenv :

$ virtualenv -p python3 venv
$ source venv/bin/activate
$ pip install -U pip

$ pip install enoslib

Configuration

Since python-grid5000 is used behind the scene, the configuration is read from a configuration file located in the home directory. It can be created with the following:

echo '
username: MYLOGIN
password: MYPASSWORD
' > ~/.python-grid5000.yaml

chmod 600 ~/.python-grid5000.yaml

With the above you can access the Grid’5000 API from you local machine aswell.

External access

If you want to control you experiment from the outside of Grid’5000 (e.g from your local machine) you can refer to the following. You can jump this section if you work from inside Grid’5000.

SSH external access

  • Solution 1: use the Grid’5000 VPN
  • Solution 2: configure you ~/.ssh/config properly:
Host *.grid5000.fr
ProxyCommand ssh -A <login>@194.254.60.33 -W "$(basename %h):%p"
User <login>
ForwardAgent yes

Accessing HTTP services inside Grid’5000

If you want to control you experiment from the outside of Grid’5000 (e.g from your local machine). For instance the Distem provider is starting a web server to handle the client requests. In order to access it propertly externally you drom your local machine can either

  • Solution 1 (general): use the Grid’5000 VPN

  • Solution 2 (HTTP traffic only): create a socks tunnel from your local machine to Grid’5000
    # on one shell
    ssh -ND 2100 access.grid5000.fr
    
    # on another shell
    export https_proxy="socks5h://localhost:2100"
    export http_proxy="socks5h://localhost:2100"
    
    # Note that browsers can work with proxy socks
    chromium-browser --proxy-server="socks5://127.0.0.1:2100" &
    
  • Solution 3 (ad’hoc): create a forwarding port tunnel

    # on one shell
    ssh -Nl 3000:paravance-42.rennes.grid5000.fr:3000 access.grid5000.fr
    
    # Now all traffic that goes on localhost:3000 is forwarded to paravance-42.rennes.grid5000.fr:3000
    

Basic example

We’ll implement a basic workflow where 2 nodes are reserved and 2 roles are described. Nodes are here put in a dedicated vlan.

Build the configuration from a dictionnary

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
from enoslib.infra.enos_g5k.provider import G5k
from enoslib.infra.enos_g5k.configuration import Configuration

import logging
import os

logging.basicConfig(level=logging.INFO)

provider_conf = {
    "resources": {
        "machines": [
            {
                "roles": ["control"],
                "cluster": "grisou",
                "nodes": 1,
                "primary_network": "n1",
                "secondary_networks": ["n2"],
            },
            {
                "roles": ["control", "compute"],
                "cluster": "grisou",
                "nodes": 1,
                "primary_network": "n1",
                "secondary_networks": ["n2"],
            },
        ],
        "networks": [
            {"id": "n1", "type": "kavlan", "roles": ["my_network"], "site": "nancy"},
            {
                "id": "n2",
                "type": "kavlan",
                "roles": ["my_second_network"],
                "site": "nancy",
            },
        ],
    }
}

# claim the resources
conf = Configuration.from_dictionnary(provider_conf)
provider = G5k(conf)
roles, networks = provider.init()

# destroy the reservation
provider.destroy()
  • You can launch the script using :

    $ python tuto_grid5000.py
    

Build the configuration programmatically

The above script can be rewritten using the programmatc API.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
from enoslib.infra.enos_g5k.provider import G5k
from enoslib.infra.enos_g5k.configuration import Configuration, NetworkConfiguration

import logging
import os

logging.basicConfig(level=logging.INFO)


# claim the resources
network = NetworkConfiguration(
    id="n1", type="kavlan",
    roles=["my_network"],
    site="rennes"
)
conf = (
    Configuration
    .from_settings(job_name="test-enoslib")
    .add_network_conf(network)
    .add_machine(
        roles=["control"],
        cluster="paravance",
        nodes=1,
        primary_network=network
    )
    .add_machine(
        roles=["control", "compute"],
        cluster="paravance",
        nodes=1,
        primary_network=network,
    )
    .finalize()
)

provider = G5k(conf)
roles, networks = provider.init()

# destroy the reservation
provider.destroy()

Note

Here we first create a network and pass its reference to each group of machine to configure the first interface of the Grid’5000 nodes.

Using the secondary interfaces

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
from enoslib.infra.enos_g5k.provider import G5k
from enoslib.infra.enos_g5k.configuration import Configuration, NetworkConfiguration

import logging


logging.basicConfig(level=logging.INFO)


SITE = "rennes"
CLUSTER = "paravance"

network = NetworkConfiguration(
    id="n1",
    type="prod",
    roles=["my_network"],
    site=SITE
)
private = NetworkConfiguration(
    id="n2",
    type="kavlan",
    roles=["private"],
    site=SITE
)

conf = (
    Configuration()
    .add_network_conf(network)
    .add_network_conf(private)
    .add_machine(
        roles=["server"],
        cluster=CLUSTER,
        nodes=1,
        primary_network=network,
        secondary_networks=[private]
    )
    .add_machine(
        roles=["client"],
        cluster=CLUSTER,
        nodes=1,
        primary_network=network,
        secondary_networks=[private]
    )
    .finalize()
)

provider = G5k(conf)
roles, networks = provider.init()
provider.destroy()

Using a custom environment

First, the description file of your environment should use resolvable URIs for the kadeploy3 server. An example of such description is the following

# myimage.desc and myimage.tgz are both located in
# the public subdirectory of rennes site of the user {{ login }}
---
name: ubuntu1804-x64-min
version: 2019052116
description: ubuntu 18.04 (bionic) - min
author: support-staff@list.grid5000.fr
visibility: public
destructive: false
os: linux
image:
file: https://api.grid5000.fr/sid/sites/rennes/public/{{ login }}/myimage.tgz
kind: tar
compression: gzip
postinstalls:
- archive: server:///grid5000/postinstalls/g5k-postinstall.tgz
compression: gzip
script: g5k-postinstall --net netplan
boot:
kernel: "/vmlinuz"
initrd: "/initrd.img"
filesystem: ext4
partition_type: 131
multipart: false

Then in the configuration of the Grid’5000 provider you can specidy the following:

job_name: test_myimage
...
env_name: https://api.grid5000.fr/sid/sites/rennes/public/{{ login }}/myimage.desc
...
resources:
...

Subnet reservation

This shows how to deal with a subnet reservation

Build the configuration from a dictionnary

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
from enoslib.infra.enos_g5k.provider import G5k
from enoslib.infra.enos_g5k.configuration import Configuration

import logging
import os

logging.basicConfig(level=logging.INFO)

provider_conf = {
    "resources": {
        "machines": [
            {
                "roles": ["control"],
                "cluster": "parapluie",
                "nodes": 1,
                "primary_network": "n1",
                "secondary_networks": [],
            }
        ],
        "networks": [
            {"id": "n1", "type": "prod", "roles": ["my_network"], "site": "rennes"},
            {
                "id": "not_linked_to_any_machine",
                "type": "slash_22",
                "roles": ["my_subnet"],
                "site": "rennes",
            },
        ],
    }
}

# claim the resources
conf = Configuration.from_dictionnary(provider_conf)
provider = G5k(conf)
roles, networks = provider.init()

# Retrieving subnet
subnet = [n for n in networks if "my_subnet" in n["roles"]]
logging.info(subnet)
# This returns the subnet information
# {
#    'roles': ['my_subnet'],
#    'start': '10.158.0.1',
#    'dns': '131.254.203.235',
#    'end': '10.158.3.254',
#    'cidr': '10.158.0.0/22',
#    'gateway': '10.159.255.254'
#    'mac_end': '00:16:3E:9E:03:FE',
#    'mac_start': '00:16:3E:9E:00:01',
# }

# destroy the reservation
provider.destroy()

Build the configuration programmatically

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
from enoslib.infra.enos_g5k.provider import G5k
from enoslib.infra.enos_g5k.configuration import Configuration, NetworkConfiguration

import logging
import os

logging.basicConfig(level=logging.INFO)


# claim the resources
prod_network = NetworkConfiguration(
    id="n1",
    type="prod",
    roles=["my_network"],
    site="rennes"
)
conf = (
    Configuration
    .from_settings(job_type="allow_classic_ssh")
    .add_network_conf(prod_network)
    .add_network(
        id="not_linked_to_any_machine",
        type="slash_22",
        roles=["my_subnet"],
        site="rennes",
    )
    .add_machine(
        roles=["control"],
        cluster="parapluie",
        nodes=1,
        primary_network=prod_network
    )
    .finalize()
)


provider = G5k(conf)
roles, networks = provider.init()

# Retrieving subnet
subnet = [n for n in networks if "my_subnet" in n["roles"]]
logging.info(subnet)
# This returns the subnet information
# {
#    'roles': ['my_subnet'],
#    'start': '10.158.0.1',
#    'dns': '131.254.203.235',
#    'end': '10.158.3.254',
#    'cidr': '10.158.0.0/22',
#    'gateway': '10.159.255.254'
#    'mac_end': '00:16:3E:9E:03:FE',
#    'mac_start': '00:16:3E:9E:00:01',
# }

# destroy the reservation
provider.destroy()

Non deploy reservation

The following shows how to deal with a non deploy reservation. Root ssh access will be granted to the nodes. For this purpose you must have a ~/.ssh/id_rsa.pub file available. All the connections will be done as root user allowing to mimic the behaviour of deploy jobs (without the kadeploy3 step). This is particularly interesting if your deployment does’t require more than one network interface.

Build the configuration from a dictionnary

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
from enoslib.infra.enos_g5k.provider import G5k
from enoslib.infra.enos_g5k.configuration import Configuration

import logging
import os


logging.basicConfig(level=logging.INFO)


provider_conf = {
    "job_type": "allow_classic_ssh",
    "job_name": "test-non-deploy",
    "resources": {
        "machines": [
            {
                "roles": ["control"],
                "cluster": "parapluie",
                "nodes": 1,
                "primary_network": "n1",
                "secondary_networks": [],
            },
            {
                "roles": ["control", "compute"],
                "cluster": "parapluie",
                "nodes": 1,
                "primary_network": "n1",
                "secondary_networks": [],
            },
        ],
        "networks": [
            {"id": "n1", "type": "prod", "roles": ["my_network"], "site": "rennes"}
        ],
    },
}

# claim the resources
conf = Configuration.from_dictionnary(provider_conf)

provider = G5k(conf)
roles, networks = provider.init()

# destroy the reservation
provider.destroy()

Build the configuration programmatically

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
from enoslib.infra.enos_g5k.provider import G5k
from enoslib.infra.enos_g5k.configuration import Configuration, NetworkConfiguration

import logging
import os


logging.basicConfig(level=logging.INFO)


# claim the resources
network = NetworkConfiguration(
    id="n1",
    type="prod",
    roles=["my_network"],
    site="rennes"
)

conf = (
    Configuration
    .from_settings(
        job_type="allow_classic_ssh",
        job_name="test-non-deploy"
    )
    .add_network_conf(network)
    .add_machine(
        roles=["control"],
        cluster="paravance",
        nodes=1,
        primary_network=network
    )
    .add_machine(
        roles=["control", "network"],
        cluster="paravance",
        nodes=1,
        primary_network=network,
    )
    .finalize()
)


try:
    provider = G5k(conf)
    roles, networks = provider.init()
    print(roles)
    print(networks)
finally:
    # destroy the reservation
    provider.destroy()

Start Virtual Machines on nodes

Hint

Note that it is now possible to use the VMonG5k provider directly. This provider will handle the reservation and deployment of virtual machines in an atomic way. The following example illustrates a mixed environment where some virtual machines are started on specific nodes of your reservation.