Working with virtualized resources on Grid’5000#

When working with bare-metal machines isn’t enough.

EnOSlib uses Providers to … provide resources. They transform an abstract resource configuration into a concrete one. To do so, they interact with an infrastructure where they get the resources from. There are different providers in EnOSlib:

Vbox/KVM to work with locally hosted virtual machines
Openstack/Chameleon to work with bare-metal resources hosted in the Chameleon platform
FiT/IOT lab to work with sensors or low profile machines
VmonG5k to work with virtual machines on Grid’5000
Distem to work with lxc containers on Grid’5000
G5k, of course

The purpose of the above is to ease the use of the platform by internalizing some of the configuration tasks (e.g automatically managing the reservation on G5k, network configuration …)

In the following we’ll cover some of the EnOSlib way of managing virtual machines on Grid’5000, docker containers or lxc containers on Grid’5000.

Website: https://discovery.gitlabpages.inria.fr/enoslib/index.html
Instant chat: https://framateam.org/enoslib
Source code: https://gitlab.inria.fr/discovery/enoslib

Prerequisites#

Make sure you’ve run the one time setup for your environment

Virtual Machines#

The VMonG5K provider which provides a quick way to start virtual machines for you on Grid’5000.

This setup is opinionated and follows these steps

First, the number of required physical machine is computed (based on cpu/ram demands) and then they are reserved
Second, a subnet is reserved (/22 or /16), which will be used as a pool of available MAC/IP
Third, the virtual machines are distributed on the physical machines and assigned a MAC/IP.
Fourth, the virtual machines are started

The following configuration wil start 10 VMs with different roles.

[ ]:

import enoslib as en

# Enable rich logging
_ = en.init_logging()


# claim the resources
conf = (
    en.VMonG5kConf
    .from_settings(job_name="enoslib_providers")
    .add_machine(
        roles=["compute"],
        cluster="paravance",
        number=8,
        flavour_desc={
            "core": 2,
            "mem": 2048
        }
    )
    .add_machine(
        roles=["controler"],
        cluster="paravance",
        number=2,
        flavour="tiny"
    )
    .finalize()
)


provider = en.VMonG5k(conf)

roles, networks = provider.init()
print(roles)
print(networks)

[ ]:

roles

[ ]:

networks

[ ]:

# VMs can take some time to be reachable, let's wait for them
en.wait_for(roles)
roles = en.sync_info(roles, networks)

[ ]:

roles

[ ]:

networks

[ ]:

results = en.run_command("nproc", roles=roles)
[(r.host,  r.stdout) for r in results]

[ ]:

provider.destroy()

Docker containers#

There’s no specific provider for manipulating Docker Container as hosts but some utility functions to help you.

most of the time you don’t need to change the internal state of docker containers (you just fire up some applications using docker container and that’s it)
so this technique cover use cases where you need to use docker container instead a bare metal machine and needs to interact with the containerized linux distribution (installing software, configuring stuffs …)

The strategy is the following: - First reserve some bare-metal machines - Install and start some docker containers - Get the representation of the docker containers as Hosts

Accessing the docker container as remote resource (beware, you need to have the docker client installed and available in your PATH) It’s not the case when running from the frontend (but we’ll likely be the case on your local machine or a G5K node)

[ ]:

import enoslib as en

en.init_logging()


prod_network = en.G5kNetworkConf(
    type="prod", roles=["my_network"], site="rennes"
)
conf = (
    en.G5kConf.from_settings(job_name="enoslib_docker", job_type=[])
    .add_network_conf(prod_network)
    .add_machine(
        roles=["control"], cluster="paravance", nodes=1, primary_network=prod_network
    )
    .finalize()
)

provider = en.G5k(conf)

# Get actual resources
roles, networks = provider.init()

We install docker using the EnOSlib’s service.

[ ]:

# Install docker
d = en.Docker(agent=roles["control"], bind_var_docker="/tmp/docker")
d.deploy()

[ ]:

# Start some containers
N = 5
with en.play_on(roles=roles) as p:
    for i in range(N):
        p.docker_container(
            name=f"mydocker-{i}",
            image="ubuntu",
            state="started",
            command="sleep 10d",
        )

[ ]:

dockers = en.get_dockers(roles=roles)

[ ]:

dockers

[ ]:

dockers[0]

EnOSlib/Ansible requires python at the destination for many operations. However minimal docker images will lickely not include python… Using the option raw=True can overcome this limitation: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/raw_module.html

[ ]:

en.run_command("hostname", roles=dockers, raw=True)

[ ]:

en.run_command("apt update && apt install -y python3", roles=dockers, raw=True)

[ ]:

# we're now good to use raw=False (the default)
en.run_command("date", roles=dockers)

[ ]:

en.run_command("apt install -y iproute2", roles=dockers)

[ ]:

results = en.run_command("ip a", roles=dockers)
for r in results:
    print(r.host)
    print("-"*20)
    print(r.stdout)

[ ]:

provider.destroy()

LXC containers with Distem#

https://distem.gitlabpages.inria.fr/

EnOSlib offers a way to use Distem on Grid’5000 using a dedicated provider. This provider will do the heavy-lifting of

reserving the physical resources needed (some machines and a subnet)
deploying a linux environment (Distem requires it)
starting the distem server and agents
makes the initial API calls to the distem server to start the wanted containers.

You need to install the distem optional dependency.

Also, since the Distem server is accessible only from inside Grid’5000, to use this provider it’s recommended to launch the script from inside Grid’5000. But if you’re adventurous and outside Grid’5000, you can try to first create a proxy SOCKS to the frontend of your choice and then set the relevant variable in your environment (and set verify: False in your ~/.python-grid5000.yaml, and … restart the kernel. Yes, notebooks are wild sometimes !)

[ ]:

from pathlib import Path

import enoslib as en

en.init_logging()

FORCE = False
CLUSTER = "paravance"

# claim the resources
conf = (
    en.DistemConf
    .from_settings(
        job_name="enoslib_distem",
        force_deploy=FORCE,
        image="file:///home/msimonin/public/distem-stretch.tgz"
    )
    .add_machine(
        roles=["server"],
        cluster=CLUSTER,
        number=1,
        flavour="large"
    )
    .add_machine(
        roles=["client"],
        cluster=CLUSTER,
        number=1,
        flavour="large"
    )
    .finalize()
)

provider = en.Distem(conf)
conf

[ ]:

roles, networks = provider.init()

[ ]:

roles

[ ]:

networks

[ ]:

roles = en.sync_info(roles, networks)

[ ]:

results = en.run_command("nproc", roles=roles)
[(r.host,  r.stdout) for r in results]

[ ]:

provider.destroy()

[ ]: