Provider APIs

Base Provider Class

Classes

Provider(provider_conf) Base class for all providers.
class enoslib.infra.provider.Provider(provider_conf)

Bases: object

Base class for all providers.

The constructor takes care of loading the configuration.

Parameters:provider_conf (BaseConfiguration) – configuration of the provider.

Methods

destroy() Abstract.
init([force_deploy]) Abstract.

The configuration object is specific to each provider and must follow the provider’s schema

destroy()

Abstract. Destroy the resources used for the deployment.

init(force_deploy=False)

Abstract. Provides resources and provisions the environment.

This calls the underlying provider and provision resources (machines and networks).

Parameters:force_deploy (bool) – Indicates that the resources must be redeployed.
Returns:roles is a dict whose key is a role and the value is the machines associated with this role. networks is the list of networks configured by the provider. see Enos_vagrant
Return type:(roles, networks) tuple

Vagrant

Vagrant Provider Class

Classes

Enos_vagrant(provider_conf) The provider to use when working with vagrant (local machine).
class enoslib.infra.enos_vagrant.provider.Enos_vagrant(provider_conf)

Bases: enoslib.infra.provider.Provider

The provider to use when working with vagrant (local machine).

The constructor takes care of loading the configuration.

Parameters:provider_conf (BaseConfiguration) – configuration of the provider.

Methods

destroy() Destroy all vagrant box involved in the deployment.
init([force_deploy]) Reserve and deploys the vagrant boxes.

The configuration object is specific to each provider and must follow the provider’s schema

destroy()

Destroy all vagrant box involved in the deployment.

init(force_deploy=False)

Reserve and deploys the vagrant boxes.

Parameters:force_deploy (bool) – True iff new machines should be started

Vagrant Schema

from ..schema import JSON_SCHEMA
from .constants import FLAVOURS, BACKENDS


SCHEMA = {
    "type": "object",
    "$schema": JSON_SCHEMA,
    "properties": {
        "backend": {"type": "string", "enum": BACKENDS},
        "box": {"type": "string"},
        "user": {"type": "string"},
        "resources": {"$ref": "#/resources"},
        "name_prefix": {"type": "string"},
        "config_extra": {"type": "string"},
    },
    "additionalProperties": False,
    "required": ["resources"],
    "resources": {
        "title": "Resource",
        "type": "object",
        "properties": {
            "machines": {"type": "array", "items": {"$ref": "#/machine"}},
            "networks": {
                "type": "array",
                "items": {"$ref": "#/network"},
                "uniqueItems": True,
            },
        },
        "additionalProperties": False,
        "required": ["machines", "networks"],
    },
    "machine": {
        "title": "Compute",
        "type": "object",
        "properties": {
            "roles": {"type": "array", "items": {"type": "string"}},
            "number": {"type": "number"},
            "name_prefix": {"type": "string"},
            "flavour": {"type": "string", "enum": list(FLAVOURS.keys())},
            "flavour_desc": {"$ref": "#/flavour_desc"},
        },
        "required": ["roles"],
        "oneOf": [{"required": ["flavour"]}, {"required": ["flavour_desc"]}],
        "additionalProperties": False,
    },
    "network": {
        "type": "object",
        "properties": {
            "cidr": {"type": "string"},
            "roles": {"type": "array", "items": {"type": "string"}},
        },
        "additionalProperties": False,
        "required": ["cidr", "roles"],
    },
    "flavour_desc": {
        "title": "Flavour",
        "type": "object",
        "properties": {"core": {"type": "number"}, "mem": {"type": "number"}},
        "required": ["core", "mem"],
        "additionalProperties": False,
    },
}

Grid5000 (G5k)

G5k Provider Class

Classes

G5k(provider_conf) The provider to use when deploying on Grid’5000.
class enoslib.infra.enos_g5k.provider.G5k(provider_conf)

Bases: enoslib.infra.provider.Provider

The provider to use when deploying on Grid’5000.

The constructor takes care of loading the configuration.

Parameters:provider_conf (BaseConfiguration) – configuration of the provider.

Methods

destroy() Destroys the jobs.
init([force_deploy, client]) Reserve and deploys the nodes according to the resources section

The configuration object is specific to each provider and must follow the provider’s schema

destroy()

Destroys the jobs.

init(force_deploy=False, client=None)

Reserve and deploys the nodes according to the resources section

In comparison to the vagrant provider, networks must be characterized as in the networks key.

Parameters:

force_deploy (bool) – True iff the environment must be redeployed

Raises:
  • MissingNetworkError – If one network is missing in comparison to what is claimed.
  • NotEnoughNodesError – If the min constraints can’t be met.

G5k Schema

# -*- coding: utf-8 -*-

from .constants import JOB_TYPES, QUEUE_TYPES, NETWORK_TYPES


SCHEMA = {
    "type": "object",
    "properties": {
        "dhcp": {"type": "boolean"},
        "force_deploy": {"type": "boolean"},
        "env_name": {"type": "string"},
        "job_name": {"type": "string"},
        "job_type": {"type": "string", "enum": JOB_TYPES},
        "key": {"type": "string"},
        "oargrid_jobids": {"type": "array", "items": {"$ref": "#/jobids"}},
        "queue": {"type": "string", "enum": QUEUE_TYPES},
        "reservation": {"type": "string"},
        "walltime": {"type": "string"},
        "resources": {"$ref": "#/resources"},
    },
    "additionalProperties": False,
    "required": ["resources"],
    "resources": {
        "title": "Resource",
        "type": "object",
        "properties": {
            "machines": {"type": "array", "items": {"$ref": "#/machine"}},
            "networks": {
                "type": "array",
                "items": {"$ref": "#/network"},
                "uniqueItems": True,
            },
        },
        "additionalProperties": False,
        "required": ["machines", "networks"],
    },
    "jobids": {"title": "JobIds", "type": "array"},
    "machine": {
        "title": "Compute",
        "type": "object",
        "properties": {
            "roles": {"type": "array", "items": {"type": "string"}},
            "cluster": {"type": "string"},
            "nodes": {"type": "number"},
            "min": {"type": "number"},
            "primary_network": {"type": "string"},
            "secondary_networks": {
                "type": "array",
                "items": {"type": "string"},
                "uniqueItems": True,
            },
        },
        "required": ["roles", "cluster", "primary_network"],
    },
    "network": {
        "type": "object",
        "properties": {
            "id": {"type": "string"},
            "type": {"enum": NETWORK_TYPES},
            "roles": {"type": "array", "items": {"type": "string"}},
            "site": {"type": "string"},
        },
        "required": ["id", "type", "roles", "site"],
    },
}
"""
Additionnal notes

Supported network types are

    - kavlan
    - kavlan-local
    - kavlan-global
    - prod
    - slash_22 (subnet reservation)
    - slash_18 (subnet reservation)

Machines must use at least one network of type prod or kavlan*. Subnets are
optional and must not be linked to any interfaces as they are a way to
claim extra ips and corresponding macs. In this case the returned network
attributes `start` and `end` corresponds to the first and last mapping of
(ip, mac).

If a key ``oargrid_jobid`` is found, the resources will be reloaded from
the corresponding oargrid job. In this case what is described under the
``resources`` key mut be compatible with the job content.

If the keys ``oar_jobid`` and ``oar_site`` are found, the resources will be
reloaded from the corresponding oar job. In this case what is described
under the ``resources`` key mut be compatible with the job content.

"""

G5k API

Classes

Resources(configuration[, client]) Class to manipulate g5k resource.
class enoslib.infra.enos_g5k.api.Resources(configuration, client=None)

Bases: object

Class to manipulate g5k resource.

This acts as an entry point to control the deployment life-cycle. A typical workflow of use would be :

Examples

Methods

configure_network()
deploy()
destroy() Destroy the associated job.
get_networks() Get the networks assoiated with the resource description.
get_roles() Get the roles associated with the hosts.
grant_root_access()
launch()
reserve()
configuration = { ... }
r = Resources(configuration)
r.reserve()
r.deploy()
r.configure_network()

Or more concisely :

configuration = { … } r = Resources(configuration) r.launch()

Note that configuration dict is not validated here, but can be through the enoslib.infra.enos_g5k.schema.validate_schema() function and thus follow the same syntax as the g5k provider.

configure_network()
deploy()
destroy()

Destroy the associated job.

get_networks()

Get the networks assoiated with the resource description.

Returns
list of tuple roles, network
get_roles()

Get the roles associated with the hosts.

Returns
dict of role -> [host]
grant_root_access()
launch()
reserve()

G5k API utils

This module is composed of helpers functions to deal with the Grid’5000 REST API.

It wraps the python-grid5000 library to provide some usual routines to interact with the platform.

Classes

Client([excluded_sites]) Wrapper of the python-grid5000 client.
ConcreteNetwork(*[, site, network, gateway, …])
ConcreteProd(*[, site, network, gateway, …]) Modelizes a Grid’5000 production network.
ConcreteSubnet(*[, site, network, gateway, …]) Modelizes a Grid’5000 subnet.
ConcreteVlan(*[, site, network, gateway, …]) Modelizes a Grid’5000 kavlan.

Functions

build_resources(jobs) Build the resources from the list of jobs.
can_start_on_cluster(nodes_status, nodes, …) Check if #nodes can be started on a given cluster.
deploy(site, nodes, options)
get_api_client() Gets the reference to the API cient (singleton).
get_api_username() Return username of client
get_cluster_interfaces(cluster[, extra_cond]) Get the network interfaces names corresponding to a criteria.
get_cluster_site(cluster) Get the site of a given cluster.
get_clusters_interfaces(clusters[, extra_cond]) Returns for each cluster the available cluster interfaces
get_clusters_sites(clusters) Get the corresponding sites of given clusters.
get_nics(cluster) Get the network cards information
grid_deploy(site, nodes, options) Deploy and wait for the deployment to be finished.
grid_destroy_from_ids(oargrid_jobids) Destroy all the jobs with corresponding ids
grid_destroy_from_name(job_name) Destroy all the jobs with a given name.
grid_reload_from_ids(oargrid_jobids) Reload all running or pending jobs of Grid’5000 from their ids
grid_reload_from_name(job_name) Reload all running or pending jobs of Grid’5000 with a given name.
set_nodes_vlan(site, nodes, interface, vlan_id) Set the interface of the nodes in a specific vlan.
submit_jobs(job_specs) Submit a job
wait_for_jobs(jobs) Waits for all the jobs to be runnning.
class enoslib.infra.enos_g5k.g5k_api_utils.Client(excluded_sites=None, **kwargs)

Bases: grid5000.Grid5000

Wrapper of the python-grid5000 client.

It accepts extra parameters to be set in the configuration file.

Constructor.

Parameters:excluded_sites (list) – sites to forget about when reloading the jobs. The primary use case was to exclude unreachable sites and allow the program to go on.
class enoslib.infra.enos_g5k.g5k_api_utils.ConcreteNetwork(*, site=None, network=None, gateway=None, dns=None, vlan_id=None, ipmac=None, nature=None, **kwargs)

Bases: object

Methods

to_nature(n_type)
static to_nature(n_type)
class enoslib.infra.enos_g5k.g5k_api_utils.ConcreteProd(*, site=None, network=None, gateway=None, dns=None, vlan_id=None, ipmac=None, nature=None, **kwargs)

Bases: enoslib.infra.enos_g5k.g5k_api_utils.ConcreteNetwork

Modelizes a Grid’5000 production network.

Methods

from_job(site)
to_enos(roles)
classmethod from_job(site)
to_enos(roles)
class enoslib.infra.enos_g5k.g5k_api_utils.ConcreteSubnet(*, site=None, network=None, gateway=None, dns=None, vlan_id=None, ipmac=None, nature=None, **kwargs)

Bases: enoslib.infra.enos_g5k.g5k_api_utils.ConcreteNetwork

Modelizes a Grid’5000 subnet.

Methods

from_job(site, subnet)
to_enos(roles)
to_nature(subnet)
classmethod from_job(site, subnet)
to_enos(roles)
static to_nature(subnet)
class enoslib.infra.enos_g5k.g5k_api_utils.ConcreteVlan(*, site=None, network=None, gateway=None, dns=None, vlan_id=None, ipmac=None, nature=None, **kwargs)

Bases: enoslib.infra.enos_g5k.g5k_api_utils.ConcreteNetwork

Modelizes a Grid’5000 kavlan.

Methods

from_job(site, vlan_id)
to_enos(roles)
to_nature(vlan_id)

Attributes

kavlan Built-in mutable sequence.
kavlan_local Built-in mutable sequence.
classmethod from_job(site, vlan_id)
kavlan = ['4', '5', '6', '7', '8', '9']
kavlan_local = ['1', '2', '3']
to_enos(roles)
static to_nature(vlan_id)
enoslib.infra.enos_g5k.g5k_api_utils.build_resources(jobs)

Build the resources from the list of jobs.

Parameters:jobs (list) – The list of python-grid5000 jobs
Returns:
nodes, networks tuple where
  • nodes is a list of all the nodes of the various reservations
  • networks is a list of all the networks of the various reservation
enoslib.infra.enos_g5k.g5k_api_utils.can_start_on_cluster(nodes_status, nodes, start, walltime)

Check if #nodes can be started on a given cluster.

This is intended to give a good enough approximation. This can be use to prefiltered possible reservation dates before submitting them on oar.

enoslib.infra.enos_g5k.g5k_api_utils.deploy(site, nodes, options)
enoslib.infra.enos_g5k.g5k_api_utils.get_api_client()

Gets the reference to the API cient (singleton).

enoslib.infra.enos_g5k.g5k_api_utils.get_api_username()

Return username of client

Returns:client’s username
enoslib.infra.enos_g5k.g5k_api_utils.get_cluster_interfaces(cluster, extra_cond=<function <lambda>>)

Get the network interfaces names corresponding to a criteria.

Note that the cluster is passed (not the individual node names), thus it is assumed that all nodes in a cluster have the same interface names same configuration. In addition to extra_cond, only the mountable and Ehernet interfaces are returned.

Parameters:
  • cluster (str) – the cluster to consider
  • extra_cond (lambda) – boolean lambda that takes the nic(dict) as parameter
enoslib.infra.enos_g5k.g5k_api_utils.get_cluster_site(cluster)

Get the site of a given cluster.

Parameters:cluster (str) – a Grid’5000 cluster
Returns:The corresponding site(str)
enoslib.infra.enos_g5k.g5k_api_utils.get_clusters_interfaces(clusters, extra_cond=<function <lambda>>)

Returns for each cluster the available cluster interfaces

Parameters:
  • clusters (str) – list of the clusters
  • extra_cond (lambda) – extra predicate to filter network card retrieved

from the API. E.g lambda nic: not nic[‘mounted’] will retrieve all the usable network cards that are not mounted by default.

Returns:dict of cluster with their associated nic names

Examples

# pseudo code
actual = get_clusters_interfaces(["paravance"])
expected = {"paravance": ["eth0", "eth1"]}
assertDictEquals(expected, actual)
enoslib.infra.enos_g5k.g5k_api_utils.get_clusters_sites(clusters)

Get the corresponding sites of given clusters.

Parameters:clusters (list) – list of the clusters (str)
Returns:dict of corresponding to the mapping cluster -> site
enoslib.infra.enos_g5k.g5k_api_utils.get_nics(cluster)

Get the network cards information

Parameters:cluster (str) – Grid’5000 cluster name
Returns:dict of nic information
enoslib.infra.enos_g5k.g5k_api_utils.grid_deploy(site, nodes, options)

Deploy and wait for the deployment to be finished.

Parameters:
  • site (str) – the site
  • nodes (list) – list of nodes (str) to depoy
  • options (dict) – option of the deployment (refer to the Grid’5000 API Specifications)
Returns:

tuple of deployed(list), undeployed(list) nodes.

enoslib.infra.enos_g5k.g5k_api_utils.grid_destroy_from_ids(oargrid_jobids)

Destroy all the jobs with corresponding ids

Parameters:oargrid_jobids (list) – the (site, oar_job_id) list of tuple identifying the jobs for each site.
enoslib.infra.enos_g5k.g5k_api_utils.grid_destroy_from_name(job_name)

Destroy all the jobs with a given name.

Parameters:job_name (str) – the job name
enoslib.infra.enos_g5k.g5k_api_utils.grid_reload_from_ids(oargrid_jobids)

Reload all running or pending jobs of Grid’5000 from their ids

Parameters:oargrid_jobids (list) – list of (site, oar_jobid) identifying the jobs on each site
Returns:The list of python-grid5000 jobs retrieved
enoslib.infra.enos_g5k.g5k_api_utils.grid_reload_from_name(job_name)

Reload all running or pending jobs of Grid’5000 with a given name.

By default all the sites will be searched for jobs with the name job_name. Using EnOSlib there can be only one job per site with name job_name.

Note that it honors the exluded_sites attribute of the client so the scan can be reduced.

Parameters:job_name (str) – the job name
Returns:The list of the python-grid5000 jobs retrieved.
Raises:EnosG5kDuplicateJobsError – if there’s several jobs with the same name on a site.
enoslib.infra.enos_g5k.g5k_api_utils.set_nodes_vlan(site, nodes, interface, vlan_id)

Set the interface of the nodes in a specific vlan.

It is assumed that the same interface name is available on the node.

Parameters:
  • site (str) – site to consider
  • nodes (list) – nodes to consider
  • interface (str) – the network interface to put in the vlan
  • vlan_id (str) – the id of the vlan
enoslib.infra.enos_g5k.g5k_api_utils.submit_jobs(job_specs)

Submit a job

Parameters:job_spec (dict) – The job specifiation (see Grid’5000 API reference)
enoslib.infra.enos_g5k.g5k_api_utils.wait_for_jobs(jobs)

Waits for all the jobs to be runnning.

Parameters:jobs (list) – list of the python-grid5000 jobs to wait for
Raises:Exception – if one of the job gets in error state.

Virtual Machines on Grid5000 (VMonG5k)

VMonG5k Provider Class

Classes

VMonG5k(provider_conf) The provider to use when deploying virtual machines on Grid’5000.
VirtualMachine(name, eui, flavour_desc, pm) Internal data structure to manipulate virtual machines.

Functions

start_virtualmachines(provider_conf, …) Starts virtualmachines on G5K.
class enoslib.infra.enos_vmong5k.provider.VMonG5k(provider_conf)

Bases: enoslib.infra.provider.Provider

The provider to use when deploying virtual machines on Grid’5000.

The constructor takes care of loading the configuration.

Parameters:provider_conf (BaseConfiguration) – configuration of the provider.

Methods

destroy() Abstract.
init([force_deploy]) Abstract.
undercloud() Gets the undercloud information (bare-metal machines).

The configuration object is specific to each provider and must follow the provider’s schema

destroy()

Abstract. Destroy the resources used for the deployment.

init(force_deploy=False)

Abstract. Provides resources and provisions the environment.

This calls the underlying provider and provision resources (machines and networks).

Parameters:force_deploy (bool) – Indicates that the resources must be redeployed.
Returns:roles is a dict whose key is a role and the value is the machines associated with this role. networks is the list of networks configured by the provider. see Enos_vagrant
Return type:(roles, networks) tuple
undercloud()

Gets the undercloud information (bare-metal machines).

class enoslib.infra.enos_vmong5k.provider.VirtualMachine(name, eui, flavour_desc, pm, extra=None, extra_devices='')

Bases: enoslib.host.Host

Internal data structure to manipulate virtual machines.

Methods

to_dict()
to_dict()
enoslib.infra.enos_vmong5k.provider.start_virtualmachines(provider_conf: enoslib.infra.enos_vmong5k.configuration.Configuration, g5k_subnets: List[Dict[Any, Any]], skip: int = 0, force_deploy: bool = False) → Tuple[MutableMapping[str, List[enoslib.host.Host]], List[Dict[Any, Any]]]

Starts virtualmachines on G5K.

This first distributes the virtual machine according to the undercloud attributes of the configuration, assign them IPs and start them. It is idempotent.

Parameters:
  • provider_conf – This is the abstract description of your overcloud (VMs). Each configuration must have its undercloud attributes filled with the undercloud machines to use. Round Robin strategy to distribute the VMs to the PMs will be used for each configuration. Mac addresses will be generated according to the g5k_subnet parameter.
  • g5k_subnets – The subnets to use. Each element is a serialization of enoslib.infra.enos_vmong5k.configuraton.NetworkConfiguration
  • skip – number of addresses to skip when distributing them to the virtual machines. This can be useful when starting incrementally the virtual machines to avoid overlaping ip assignements between iterations.
  • force_deploy (boolean) – controls whether the virtual machines should be restarted from scratch.

Examples

Returns:(roles, networks) tuple

VMonG5k Schema

from .constants import FLAVOURS


from enoslib.infra.enos_g5k.constants import QUEUE_TYPES, SUBNET_TYPES


STRATEGY = ["copy", "cow"]

SCHEMA = {
    "description": "VMonG5k schema.",
    "type": "object",
    "properties": {
        "resources": {"$ref": "#/resources"},
        "enable_taktuk": {
            "type": "boolean",
            "description": "Copy the base image on remote hosts using taktuk",
        },
        "force_deploy": {
            "type": "boolean",
            "description": "Remove and restart all virtual machines",
        },
        "gateway": {
            "type": "boolean",
            "description": "Enable access to virtual machines from outside Grid'5000",
        },
        "job_name": {"type": "string", "description": "Sets the job name"},
        "queue": {
            "type": "string",
            "enum": QUEUE_TYPES,
            "description": "Grid'5000 queue to use",
        },
        "walltime": {"type": "string", "description": "Job duration"},
        "image": {
            "type": "string",
            "description": "Path to the base image on the reserved nodes",
        },
        "skip": {"type": "number", "description": "Skip this number of IPs"},
        "strategy": {
            "type": "string",
            "enum": STRATEGY,
            "description": "Base image strategy (cow, copy)",
        },
        "subnet_type": {
            "type": "string",
            "enum": SUBNET_TYPES,
            "description": "Subnet type to use (/16, /22)",
        },
        "working_dir": {
            "type": "string",
            "description": "Remote directory that hold the image",
        },
    },
    "additionalProperties": False,
    "required": ["resources"],
    "resources": {
        "title": "Resource",
        "type": "object",
        "properties": {
            "machines": {"type": "array", "items": {"$ref": "#/machine"}},
            "networks": {"type": "array", "items": {"type": "string"}},
        },
        "additionalProperties": False,
        "required": ["machines", "networks"],
    },
    "machine": {
        "description": "Machine description",
        "title": "Compute",
        "type": "object",
        "properties": {
            "roles": {
                "type": "array",
                "items": {"type": "string"},
                "description": "EnOSlib's roles",
            },
            "number": {"type": "number", "description": "Number of machines to start"},
            "flavour": {
                "type": "string",
                "enum": list(FLAVOURS.keys()),
                "description": "Predefined flavour name",
            },
            "flavour_desc": {
                "$ref": "#/flavour_desc",
                "description": "Custom flavour description",
            },
            "cluster": {
                "type": "string",
                "description": "Grid'5000 cluster for the undercloud",
            },
            "undercloud": {
                "type": "array",
                "items": {"type": "object"},
                "description": "(optional)List of Host where the VM should be started.",
            },
            "extra_devices": {
                "type": "string",
                "description": "Libvirt XML description for extra devices (e.g disks).",
            },
        },
        "required": ["roles"],
        "additionalProperties": False,
    },
    "flavour_desc": {
        "description": "Custom flavour for a virtual machine.",
        "title": "Flavour",
        "type": "object",
        "properties": {
            "core": {"type": "number", "description": "number of cores"},
            "mem": {"type": "number", "description": "memory size in MB"},
            "disk": {"type": "number", "description": "disk size in GB"},
        },
        "required": ["core", "mem"],
        "additionalProperties": False,
    },
}

Containers on Grid5000 (Distem)

Distem Provider Class

Classes

Distem(provider_conf) Use Distem on G5k

Functions

distem_bootstrap(roles, path_sshkeys) Bootstrap distem on G5k nodes
start_containers(g5k_roles, provider_conf, …) Starts containers on G5K.
write_ssh_keys(path)
class enoslib.infra.enos_distem.provider.Distem(provider_conf)

Bases: enoslib.infra.provider.Provider

Use Distem on G5k

The constructor takes care of loading the configuration.

Parameters:provider_conf (BaseConfiguration) – configuration of the provider.

Methods

destroy() Abstract.
init([force_deploy]) Abstract.

The configuration object is specific to each provider and must follow the provider’s schema

destroy()

Abstract. Destroy the resources used for the deployment.

init(force_deploy=False)

Abstract. Provides resources and provisions the environment.

This calls the underlying provider and provision resources (machines and networks).

Parameters:force_deploy (bool) – Indicates that the resources must be redeployed.
Returns:roles is a dict whose key is a role and the value is the machines associated with this role. networks is the list of networks configured by the provider. see Enos_vagrant
Return type:(roles, networks) tuple
enoslib.infra.enos_distem.provider.distem_bootstrap(roles, path_sshkeys)

Bootstrap distem on G5k nodes

Args :
roles (dict): physical machines to start containers on. path_sshkeys (dict): ssh keys paths
Return :
distem (class): distem client
enoslib.infra.enos_distem.provider.start_containers(g5k_roles, provider_conf, g5k_subnets)

Starts containers on G5K.

Parameters:
  • g5k_roles (dict) – physical machines to start the containers on.
  • provider_conf (Configuration) – enoslib.infra.enos_distem.configuraton.Configuration This is the abstract description of your overcloud (containers). Each configuration has a its undercloud attributes filled with the undercloud machines to use. Round Robin strategy to distribute the containers to the PMs will be used for each configuration. Mac addresses will be generated according to the g5k_subnet parameter.
  • g5k_subnets (list) – The subnets to use. Each element is a serialization of enoslib.infra.enos_distem.configuraton.NetworkConfiguration
Returns:

(roles, networks) tuple

enoslib.infra.enos_distem.provider.write_ssh_keys(path)

Distem Schema

from .constants import FLAVOURS


from enoslib.infra.enos_g5k.constants import QUEUE_TYPES


STRATEGY = ["copy", "cow"]

SCHEMA = {
    "type": "object",
    "properties": {
        "resources": {"$ref": "#/resources"},
        "job_name": {"type": "string"},
        "queue": {"type": "string", "enum": QUEUE_TYPES},
        "walltime": {"type": "string"},
        "image": {"type": "string"},
        ###
        "force_deploy": {"type": "boolean"},
    },
    "additionalProperties": False,
    "required": ["resources", "image"],
    "resources": {
        "title": "Resource",
        "type": "object",
        "properties": {
            "machines": {"type": "array", "items": {"$ref": "#/machine"}},
            "networks": {"type": "array", "items": {"type": "string"}},
        },
        "additionalProperties": False,
        "required": ["machines", "networks"],
    },
    "machine": {
        "title": "Compute",
        "type": "object",
        "properties": {
            "roles": {"type": "array", "items": {"type": "string"}},
            "number": {"type": "number"},
            "flavour": {"type": "string", "enum": list(FLAVOURS.keys())},
            "flavour_desc": {"$ref": "#/flavour_desc"},
            "cluster": {"type": "string"},
            "undercloud": {"type": "array", "items": {"type": "object"}},
        },
        "required": ["roles"],
        "oneOf": [{"required": ["flavour"]}, {"required": ["flavour_desc"]}],
        "additionalProperties": False,
    },
    "flavour_desc": {
        "title": "Flavour",
        "type": "object",
        "properties": {"core": {"type": "number"}, "mem": {"type": "number"}},
        "required": ["core", "mem"],
        "additionalProperties": False,
    },
}

Static

Static Provider Class

Classes

Static(provider_conf) The Static provider class.
class enoslib.infra.enos_static.provider.Static(provider_conf)

Bases: enoslib.infra.provider.Provider

The Static provider class.

This class is used when one has already machines and network configured. This is usefull when there si no provider class corresponding the user testbed. To use it the user must build a configuration object that reflects the exact settings of his machines and networks.

The constructor takes care of loading the configuration.

Parameters:provider_conf (BaseConfiguration) – configuration of the provider.

Methods

destroy() Abstract.
init([force_deploy]) Abstract.

The configuration object is specific to each provider and must follow the provider’s schema

destroy()

Abstract. Destroy the resources used for the deployment.

init(force_deploy=False)

Abstract. Provides resources and provisions the environment.

This calls the underlying provider and provision resources (machines and networks).

Parameters:force_deploy (bool) – Indicates that the resources must be redeployed.
Returns:roles is a dict whose key is a role and the value is the machines associated with this role. networks is the list of networks configured by the provider. see Enos_vagrant
Return type:(roles, networks) tuple

Static Schema

SCHEMA = {
    "type": "object",
    "properties": {"resources": {"$ref": "#/resources"}},
    "additionalProperties": True,
    "required": ["resources"],
    "resources": {
        "title": "Resource",
        "type": "object",
        "properties": {
            "machines": {"type": "array", "items": {"$ref": "#/machine"}},
            "networks": {
                "type": "array",
                "items": {"$ref": "#/network"},
                "uniqueItems": True,
            },
        },
        "additionalProperties": False,
        "required": ["machines", "networks"],
    },
    "machine": {
        "title": "Compute",
        "type": "object",
        "properties": {
            "roles": {"type": "array", "items": {"type": "string"}},
            "address": {"type": "string"},
            "alias": {"type": "string"},
            "user": {"type": "string"},
            "keyfile": {"type": "string"},
            "port": {"type": "number"},
            "extra": {"type": "object"},
        },
        "required": ["roles", "address"],
    },
    "network": {
        "type": "object",
        "properties": {
            "roles": {"type": "array", "items": {"type": "string"}},
            # TODO(msimonin): validate the ip schema
            "start": {"type": "string"},
            "end": {"type": "string"},
            "cidr": {"type": "string"},
            "gateway": {"type": "string"},
            "dns": {"type": "string"},
        },
        "required": ["roles", "start", "end", "cidr", "gateway", "dns"],
    },
}

Openstack

Openstack Provider Class

Openstack Schema

SCHEMA = {
    "type": "object",
    "properties": {
        "resources": {"$ref": "#/resources"},
        # Mandatory keys
        "key_name": {"type": "string"},
        "image": {"type": "string"},
        "user": {"type": "string"},
        # optional keys
        "allocation_pool": {"$ref": "#/os_allocation_pool"},
        "configure_network": {"type": "boolean"},
        "dns_nameservers": {"type": "array", "items": {"type": "string"}},
        "gateway": {"type": "boolean"},
        "gateway_user": {"type": "string"},
        "network": {"$ref": "#/os_network"},
        "subnet": {"$ref": "#/os_subnet"},
        "prefix": {"type": "string"},
    },
    "additionalProperties": False,
    "required": ["resources", "key_name", "image", "user"],
    "os_allocation_pool": {
        "title": "OSallocationPool",
        "type": "object",
        "properties": {"start": {"type": "string"}, "end": {"type": "string"}},
        "required": ["start", "end"],
        "additionalProperties": False,
    },
    "os_network": {
        "title": "OSNetwork",
        "type": "object",
        "properties": {"name": {"type": "string"}},
        "required": ["name"],
        "additionalProperties": False,
    },
    "os_subnet": {
        "title": "OSSubnet",
        "type": "object",
        "properties": {"name": {"type": "string"}, "cidr": {"type": "string"}},
        "required": ["name", "cidr"],
        "additionalProperties": False,
    },
    "resources": {
        "title": "Resource",
        "type": "object",
        "properties": {
            "machines": {"type": "array", "items": {"$ref": "#/machine"}},
            "networks": {"type": "array", "items": {"type": "string"}},
        },
        "additionalProperties": False,
        "required": ["machines", "networks"],
    },
    "machine": {
        "title": "Compute",
        "type": "object",
        "properties": {
            "roles": {"type": "array", "items": {"type": "string"}},
            "flavour": {"type": "string"},
            "number": {"type": "number"},
        },
        "required": ["roles", "number", "flavour"],
    },
}

Chameleon

Chameleon(kvm) Provider Class

Chameleon(kvm) Schema

SCHEMA = {
    "type": "object",
    "properties": {
        "resources": {"$ref": "#/resources"},
        "key_name": {"type": "string"},
        # everything is optionnal
        "image": {"type": "string"},
        "user": {"type": "string"},
        "allocation_pool": {"$ref": "#/os_allocation_pool"},
        "configure_network": {"type": "boolean"},
        "dns_nameservers": {"type": "array", "items": {"type": "string"}},
        "gateway": {"type": "boolean"},
        "gateway_user": {"type": "string"},
        "network": {"$ref": "#/os_network"},
        "subnet": {"$ref": "#/os_subnet"},
        "prefix": {"type": "string"},
    },
    "additionalProperties": True,
    "required": ["resources", "key_name"],
    "os_allocation_pool": {
        "title": "OSallocationPool",
        "type": "object",
        "properties": {"start": {"type": "string"}, "end": {"type": "string"}},
        "required": ["start", "end"],
        "additionalProperties": False,
    },
    "os_network": {
        "title": "OSNetwork",
        "type": "object",
        "properties": {"name": {"type": "string"}},
        "required": ["name"],
        "additionalProperties": False,
    },
    "os_subnet": {
        "title": "OSSubnet",
        "type": "object",
        "properties": {"name": {"type": "string"}, "cidr": {"type": "string"}},
        "required": ["name"],
        "additionalProperties": False,
    },
    "resources": {
        "title": "Resource",
        "type": "object",
        "properties": {
            "machines": {"type": "array", "items": {"$ref": "#/machine"}},
            "networks": {"type": "array", "items": {"type": "string"}},
        },
        "additionalProperties": False,
        "required": ["machines", "networks"],
    },
    "machine": {
        "title": "Compute",
        "type": "object",
        "properties": {
            "roles": {"type": "array", "items": {"type": "string"}},
            "flavour": {"type": "string"},
            "number": {"type": "number"},
        },
        "required": ["roles", "number", "flavour"],
    },
}

Chameleon(bare metal) Provider Class

Chameleon(bare metal) schema

SCHEMA = {
    "type": "object",
    "properties": {
        "resources": {"$ref": "#/resources"},
        "key_name": {"type": "string"},
        # everything is optionnal
        "image": {"type": "string"},
        "user": {"type": "string"},
        "allocation_pool": {"$ref": "#/os_allocation_pool"},
        "configure_network": {"type": "boolean"},
        "dns_nameservers": {"type": "array", "items": {"type": "string"}},
        "gateway": {"type": "boolean"},
        "gateway_user": {"type": "string"},
        "network": {"$ref": "#/os_network"},
        "subnet": {"$ref": "#/os_subnet"},
        "prefix": {"type": "string"},
    },
    "additionalProperties": True,
    "required": ["resources", "key_name"],
    "os_allocation_pool": {
        "title": "OSallocationPool",
        "type": "object",
        "properties": {"start": {"type": "string"}, "end": {"type": "string"}},
        "required": ["start", "end"],
        "additionalProperties": False,
    },
    "os_network": {
        "title": "OSNetwork",
        "type": "object",
        "properties": {"name": {"type": "string"}},
        "required": ["name"],
        "additionalProperties": False,
    },
    "os_subnet": {
        "title": "OSSubnet",
        "type": "object",
        "properties": {"name": {"type": "string"}, "cidr": {"type": "string"}},
        "required": ["name"],
        "additionalProperties": False,
    },
    "resources": {
        "title": "Resource",
        "type": "object",
        "properties": {
            "machines": {"type": "array", "items": {"$ref": "#/machine"}},
            "networks": {"type": "array", "items": {"type": "string"}},
        },
        "additionalProperties": False,
        "required": ["machines", "networks"],
    },
    "machine": {
        "title": "Compute",
        "type": "object",
        "properties": {
            "roles": {"type": "array", "items": {"type": "string"}},
            "flavour": {"type": "string"},
            "number": {"type": "number"},
        },
        "required": ["roles", "number", "flavour"],
    },
}