The Cluster class

Using DC/OS E2E usually involves creating one or more Clusters. A cluster is created using a “backend”, which might be Docker or a cloud provider for example. It is also possible to point DC/OS E2E to existing nodes. A Cluster object is then used to interact with the DC/OS cluster.

class dcos_e2e.cluster.Cluster(cluster_backend, masters=1, agents=1, public_agents=1)

Create a DC/OS cluster.

Parameters

  • cluster_backend (ClusterBackend) – The backend to use for the cluster.

  • masters (int) – The number of master nodes to create.

  • agents (int) – The number of agent nodes to create.

  • public_agents (int) – The number of public agent nodes to create.

Choosing a Backend

See Backends for a backend to use for cluster_backend.

Creating a Cluster from Existing Nodes

It is possible to create a Cluster from existing nodes. Clusters created with this method cannot be destroyed by DC/OS E2E. It is assumed that DC/OS is already up and running on the given Nodes and installing DC/OS is not supported.

classmethod Cluster.from_nodes(masters, agents, public_agents)

Create a cluster from existing nodes.

Parameters

  • masters (Set[Node]) – The master nodes in an existing cluster.

  • agents (Set[Node]) – The agent nodes in an existing cluster.

  • public_agents (Set[Node]) – The public agent nodes in an existing cluster.

Return type

Cluster

Returns

A cluster object with the nodes of an existing cluster.

Installing DC/OS

See how to use DC/OS Enterprise with DC/OS E2E.

Cluster.install_dcos_from_path(dcos_installer, dcos_config, ip_detect_path, output=<Output.CAPTURE: 2>, files_to_copy_to_genconf_dir=())

Installs DC/OS using the DC/OS advanced installation method.

Parameters

  • dcos_installer (Path) – The Path to a local installer to install DC/OS from.

  • dcos_config (Dict[str, Any]) – The contents of the DC/OS config.yaml.

  • ip_detect_path (Path) – The path to a ip-detect script that will be used when installing DC/OS.

  • files_to_copy_to_genconf_dir (Iterable[Tuple[Path, Path]]) – Pairs of host paths to paths on the installer node. These are files to copy from the host to the installer node before installing DC/OS.

  • output (Output) – What happens with stdout and stderr.

Return type

None

Cluster.install_dcos_from_url(dcos_installer, dcos_config, ip_detect_path, output=<Output.CAPTURE: 2>, files_to_copy_to_genconf_dir=())

Installs DC/OS using the DC/OS advanced installation method.

Parameters

  • dcos_installer (str) – A URL pointing to an installer to install DC/OS from.

  • dcos_config (Dict[str, Any]) – The contents of the DC/OS config.yaml.

  • ip_detect_path (Path) – The path to a ip-detect script that will be used when installing DC/OS.

  • files_to_copy_to_genconf_dir (Iterable[Tuple[Path, Path]]) – Pairs of host paths to paths on the installer node. These are files to copy from the host to the installer node before installing DC/OS.

  • output (Output) – What happens with stdout and stderr.

Return type

None

Destroying a Cluster

Clusters have a destroy() method. This can be called manually, or Clusters can be used as context managers. In this case the cluster will be destroyed when exiting the context manager.

with Cluster(backend=Docker(), masters=3, agents=2):
    pass
Cluster.destroy()

Destroy all nodes in the cluster.

Return type

None

Upgrading a Cluster

It is possible to upgrade a Clusters DC/OS installation.

Cluster.upgrade_dcos_from_path(dcos_installer, dcos_config, ip_detect_path, output=<Output.CAPTURE: 2>, files_to_copy_to_genconf_dir=())

Upgrade DC/OS.

Parameters

  • dcos_installer (Path) – The Path to a local installer or a str to which is a URL pointing to an installer to install DC/OS from.

  • dcos_config (Dict[str, Any]) – The DC/OS configuration to use.

  • ip_detect_path (Path) – The path to a ip-detect script that will be used when installing DC/OS.

  • files_to_copy_to_genconf_dir (Iterable[Tuple[Path, Path]]) – Pairs of host paths to paths on the installer node. These are files to copy from the host to the installer node before installing DC/OS.

  • output (Output) – What happens with stdout and stderr.

Return type

None

Cluster.upgrade_dcos_from_url(dcos_installer, dcos_config, ip_detect_path, output=<Output.CAPTURE: 2>, files_to_copy_to_genconf_dir=())

Upgrade DC/OS.

Parameters

  • dcos_installer (str) – A URL pointing to an installer to upgrade DC/OS from.

  • dcos_config (Dict[str, Any]) – The DC/OS configuration to use.

  • ip_detect_path (Path) – The path to a ip-detect script that will be used when installing DC/OS.

  • files_to_copy_to_genconf_dir (Iterable[Tuple[Path, Path]]) – Pairs of host paths to paths on the installer node. These are files to copy from the host to the installer node before installing DC/OS.

  • output (Output) – What happens with stdout and stderr.

Return type

None

Waiting for DC/OS

Depending on the hardware and the backend, DC/OS can take some time to install. The methods to wait for DC/OS repeatedly poll the cluster until services are up. Choose the wait_for_dcos_oss() or wait_for_dcos_ee() as appropriate.

Cluster.wait_for_dcos_oss(http_checks=True)

Wait until the DC/OS OSS boot process has completed.

Parameters

http_checks (bool) – Whether or not to wait for checks which involve HTTP. If this is False, this function may return before DC/OS is fully ready. This is useful in cases where an HTTP connection cannot be made to the cluster. For example, this is useful on macOS without a VPN set up.

Raises

dcos_e2e.exceptions.DCOSTimeoutError – Raised if cluster components did not become ready within one hour.

Return type

None

Cluster.wait_for_dcos_ee(superuser_username, superuser_password, http_checks=True)

Wait until the DC/OS Enterprise boot process has completed.

Parameters

  • superuser_username (str) – Username of a user with superuser privileges.

  • superuser_password (str) – Password of a user with superuser privileges.

  • http_checks (bool) – Whether or not to wait for checks which involve HTTP. If this is False, this function may return before DC/OS is fully ready. This is useful in cases where an HTTP connection cannot be made to the cluster. For example, this is useful on macOS without a VPN set up.

Raises

dcos_e2e.exceptions.DCOSTimeoutError – Raised if cluster components did not become ready within one hour.

Return type

None

Running Integration Tests

It is possible to easily run DC/OS integration tests on a cluster. See how to run tests on DC/OS Enterprise.

with Cluster(backend=Docker()):
    cluster.run_with_test_environment(args=['pytest', '-k', 'mesos'])
Cluster.run_with_test_environment(args, env=None, output=<Output.CAPTURE: 2>, tty=False, node=None, transport=None)

Run a command on a node using the Mesosphere test environment.

Parameters

  • args (List[str]) – The command to run on the node.

  • env (Optional[Dict[str, Any]]) – Environment variables to be set on the node before running the command. On enterprise clusters, DCOS_LOGIN_UNAME and DCOS_LOGIN_PW must be set.

  • output (Output) – What happens with stdout and stderr.

  • node (Optional[Node]) – The node to run the given command on. if not given, an arbitrary master node is used.

  • tty (bool) – If True, allocate a pseudo-tty. This means that the users terminal is attached to the streams of the process. This means that the values of stdout and stderr will not be in the returned subprocess.CompletedProcess.

  • transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

Return type

CompletedProcess

Returns

The result of the given command.

Raises

subprocess.CalledProcessError – If the command fails.