Cluster Nodes

Clusters are made of Nodes. The Node interface is backend agnostic.

Nodes are generally used to run commands.

Nodes are either manually constructed in order to create a from_nodes(), or they are retrieved from an existing Cluster.

Tools for managing DC/OS cluster nodes.

class dcos_e2e.node.Role

Roles of DC/OS nodes.

MASTER = 'master'
AGENT = 'slave'
PUBLIC_AGENT = 'slave_public'
class dcos_e2e.node.DCOSVariant

Variant of DC/OS.

OSS = 1
ENTERPRISE = 2
class dcos_e2e.node.DCOSBuildInfo(version, commit, variant)

DC/OS build info object.

Parameters

  • version (str) – A version of DC/OS.

  • commit (str) – A commit hash of DC/OS.

  • variant (DCOSVariant) – A DC/OS variant.

version

A version of DC/OS.

commit

A commit hash of DC/OS.

variant

A DC/OS variant.

class dcos_e2e.node.Transport

Transports for communicating with nodes.

SSH = 1
DOCKER_EXEC = 2
class dcos_e2e.node.Output

Output capture options for running commands.

When using LOG_AND_CAPTURE, stdout and stderr are merged into stdout.

LOG_AND_CAPTURE

Log output at the debug level. If the code returns a subprocess.CompletedProcess, the stdout and stderr will be contained in the return value. However, they will be merged into stdout.

CAPTURE

Capture stdout and stderr. If the code returns a subprocess.CompletedProcess, the stdout and stderr will be contained in the return value.

NO_CAPTURE

Do not capture stdout or stderr.

LOG_AND_CAPTURE = 1
CAPTURE = 2
NO_CAPTURE = 3
class dcos_e2e.node.Node(public_ip_address, private_ip_address, default_user, ssh_key_path, default_transport=<Transport.SSH: 1>)
Parameters

  • public_ip_address (IPv4Address) – The public IP address of the node.

  • private_ip_address (IPv4Address) – The IP address used by the DC/OS component running on this node.

  • default_user (str) – The default username to use for connections.

  • ssh_key_path (Path) – The path to an SSH key which can be used to SSH to the node as the default_user user. The file must only have permissions to be read by (and optionally written to) the owner.

  • default_transport (Transport) – The transport to use for communicating with nodes.

public_ip_address

The public IP address of the node.

private_ip_address

The IP address used by the DC/OS component running on this node.

default_user

The default username to use for connections.

default_transport

The transport used to communicate with the node.

install_dcos_from_url(dcos_installer, dcos_config, ip_detect_path, role, files_to_copy_to_genconf_dir=(), user=None, output=<Output.CAPTURE: 2>, transport=None)

Install DC/OS in a platform-independent way by using the advanced installation method as described at https://docs.d2iq.com/mesosphere/dcos/2.1/installing/production/deploying-dcos/installation/.

The documentation describes using a “bootstrap” node, so that only one node downloads and extracts the installer. This method is less efficient on a multi-node cluster, as it does not use a bootstrap node. Instead, the installer is put on this node and then extracted on this node, and then DC/OS is installed.

This creates a folder in /dcos-install-dir on this node which contains the DC/OS installation files that can be removed safely after the DC/OS installation has finished.

Parameters

  • dcos_installer (str) – A URL pointing to an installer to install DC/OS from.

  • dcos_config (Dict[str, Any]) – The contents of the DC/OS config.yaml.

  • ip_detect_path (Path) – The path to the ip-detect script to use for installing DC/OS.

  • role (Role) – The desired DC/OS role for the installation.

  • user (Optional[str]) – The username to communicate as. If None then the default_user is used instead.

  • output (Output) – What happens with stdout and stderr.

  • transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

  • files_to_copy_to_genconf_dir (Iterable[Tuple[Path, Path]]) – Pairs of host paths to paths on the installer node. These are files to copy from the host to the installer node before installing DC/OS.

Return type

None

install_dcos_from_path(dcos_installer, dcos_config, ip_detect_path, role, files_to_copy_to_genconf_dir=(), user=None, output=<Output.CAPTURE: 2>, transport=None)

Install DC/OS in a platform-independent way by using the advanced installation method as described at https://docs.d2iq.com/mesosphere/dcos/2.1/installing/production/deploying-dcos/installation/.

The documentation describes using a “bootstrap” node, so that only one node downloads and extracts the installer. This method is less efficient on a multi-node cluster, as it does not use a bootstrap node. Instead, the installer is put on this node and then extracted on this node, and then DC/OS is installed.

This creates a folder in /dcos-install-dir on this node which contains the DC/OS installation files that can be removed safely after the DC/OS installation has finished.

Parameters

  • dcos_installer (Path) – The Path to a local installer to install DC/OS from.

  • dcos_config (Dict[str, Any]) – The contents of the DC/OS config.yaml.

  • ip_detect_path (Path) – The path to the ip-detect script to use for installing DC/OS.

  • role (Role) – The desired DC/OS role for the installation.

  • user (Optional[str]) – The username to communicate as. If None then the default_user is used instead.

  • output (Output) – What happens with stdout and stderr.

  • transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

  • files_to_copy_to_genconf_dir (Iterable[Tuple[Path, Path]]) – Pairs of host paths to paths on the installer node. These are files to copy from the host to the installer node before installing DC/OS.

Return type

None

upgrade_dcos_from_url(dcos_installer, dcos_config, ip_detect_path, role, files_to_copy_to_genconf_dir=(), user=None, output=<Output.CAPTURE: 2>, transport=None)

Upgrade DC/OS on this node. This follows the steps in https://docs.d2iq.com/mesosphere/dcos/2.1/installing/production/upgrading/.

Parameters

  • dcos_installer (str) – A URL pointing to an installer to install DC/OS from.

  • dcos_config (Dict[str, Any]) – The contents of the DC/OS config.yaml.

  • ip_detect_path (Path) – The path to the ip-detect script to use for installing DC/OS.

  • role (Role) – The desired DC/OS role for the installation.

  • user (Optional[str]) – The username to communicate as. If None then the default_user is used instead.

  • output (Output) – What happens with stdout and stderr.

  • transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

  • files_to_copy_to_genconf_dir (Iterable[Tuple[Path, Path]]) – Pairs of host paths to paths on the installer node. These are files to copy from the host to the installer node before installing DC/OS.

Return type

None

upgrade_dcos_from_path(dcos_installer, dcos_config, ip_detect_path, role, files_to_copy_to_genconf_dir=(), user=None, output=<Output.CAPTURE: 2>, transport=None)

Upgrade DC/OS on this node. This follows the steps in https://docs.d2iq.com/mesosphere/dcos/2.1/installing/production/upgrading/.

Parameters

  • dcos_installer (Path) – The Path to a local installer to upgrade DC/OS from.

  • dcos_config (Dict[str, Any]) – The contents of the DC/OS config.yaml.

  • ip_detect_path (Path) – The path to the ip-detect script to use for installing DC/OS.

  • role (Role) – The desired DC/OS role for the installation.

  • user (Optional[str]) – The username to communicate as. If None then the default_user is used instead.

  • output (Output) – What happens with stdout and stderr.

  • transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

  • files_to_copy_to_genconf_dir (Iterable[Tuple[Path, Path]]) – Pairs of host paths to paths on the installer node. These are files to copy from the host to the installer node before installing DC/OS.

Return type

None

run(args, user=None, output=<Output.CAPTURE: 2>, env=None, shell=False, tty=False, transport=None, sudo=False)

Run a command on this node the given user.

Parameters

  • args (List[str]) – The command to run on the node.

  • user (Optional[str]) – The username to communicate as. If None then the default_user is used instead.

  • output (Output) – What happens with stdout and stderr.

  • env (Optional[Dict[str, Any]]) – Environment variables to be set on the node before running the command. A mapping of environment variable names to values.

  • shell (bool) – If False (the default), each argument is passed as a literal value to the command. If True, the command line is interpreted as a shell command, with a special meaning applied to some characters (e.g. $, &&, >). This means the caller must quote arguments if they may contain these special characters, including whitespace.

  • tty (bool) – If True, allocate a pseudo-tty. This means that the users terminal is attached to the streams of the process. When using a TTY, different transports may use different line endings.

  • transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

  • sudo (bool) – Whether to use “sudo” to run commands.

Return type

CompletedProcess

Returns

The representation of the finished process.

Raises

subprocess.CalledProcessError – The process exited with a non-zero code.

popen(args, user=None, env=None, shell=False, transport=None)

Open a pipe to a command run on a node as the given user.

Parameters

  • args (List[str]) – The command to run on the node.

  • user (Optional[str]) – The user to open a pipe for a command for over. If None the default_user is used instead.

  • env (Optional[Dict[str, Any]]) – Environment variables to be set on the node before running the command. A mapping of environment variable names to values.

  • shell (bool) – If False (the default), each argument is passed as a literal value to the command. If True, the command line is interpreted as a shell command, with a special meaning applied to some characters (e.g. $, &&, >). This means the caller must quote arguments if they may contain these special characters, including whitespace.

  • transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

Return type

Popen

Returns

The pipe object attached to the specified process.

send_file(local_path, remote_path, user=None, transport=None, sudo=False)

Copy a file to this node.

Parameters

  • local_path (Path) – The path on the host of the file to send.

  • remote_path (Path) – The path on the node to place the file.

  • user (Optional[str]) – The name of the remote user to send the file. If None, the default_user is used instead.

  • transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

  • sudo (bool) – Whether to use sudo to create the directory which holds the remote file.

Return type

None

download_file(remote_path, local_path, transport=None)

Download a file from this node.

Parameters

  • remote_path (Path) – The path on the node to download the file from.

  • local_path (Path) – The path on the host to download the file to.

  • transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

Raises

ValueError – The remote_path does not exist. The local_path is an existing file.

Return type

None

dcos_build_info(transport=None)

Download a file from this node.

Parameters

transport (Optional[Transport]) – The transport to use for communicating with nodes. If None, the Node’s default_transport is used.

Raises

DCOSNotInstalledError – The DC/OS build information is not available because DC/OS is not installed on the Node.

Return type

DCOSBuildInfo