`dcos-aws` CLI¶

The dcos-aws CLI allows you to create and manage open source DC/OS and DC/OS Enterprise clusters on AWS EC2 instances.

A typical CLI workflow for open source DC/OS may look like the following. Install the CLI, then create and manage a cluster:

# Fix issues shown by dcos-aws doctor
$ dcos-aws doctor
$ dcos-aws create https://downloads.dcos.io/dcos/stable/dcos_generate_config.sh --variant oss
default
$ dcos-aws wait
$ dcos-aws run --sync-dir /path/to/dcos/checkout pytest -k test_tls
...

Each of these and more are described in detail below.

Requirements ¶

Amazon Web Services ¶

An Amazon Web Services account with sufficient funds must be available.

The AWS credentials for the account must be present either in the environment as environment variables or in the default file system location under ~/.aws/credentials with a AWS profile in the environment referencing those credentials.

The Mesosphere internal AWS tool maws automatically stores account specific temporary AWS credentials in the default file system location and exports the corresponding profile into the environment. After logging in with maws clusters can be launched using the AWS backend.

For CI deployments long lived credentials are preferred. It is recommended to use the environment variables method for AWS credentials in that case.

The environment variables are set as follows:

export AWS_ACCESS_KEY_ID=<aws_access_key_id>
export AWS_SECRET_ACCESS_KEY=<aws_secret_access_key>

The EC2 instances launched by the AWS backend will bring about costs in the order of 24 ct per instance, assuming the fixed cluster lifetime of two hours and m4.large EC2 instances.

`ssh`¶

The ssh command must be available.

Operating System ¶

The AWS backend has been tested on macOS and on Linux.

It is not expected that it will work out of the box with Windows, see issue QUALITY-1771.

If your operating system is not supported, it may be possible to use Vagrant, or another Linux virtual machine.

`dcos-aws doctor`¶

DC/OS E2E comes with the dcos-aws doctor command. Run this command to check your system for common causes of problems.

Installing the CLI ¶

The CLI can be installed with Homebrew on macOS or Linuxbrew on Linux. The library and CLI can be installed together with pip on any Linux and macOS.

CLI on macOS With Homebrew ¶

To install the CLI on macOS, install Homebrew.

Then install the latest stable version:

brew install https://raw.githubusercontent.com/dcos/dcos-e2e/master/dcose2e.rb

To upgrade from an older version, run the following command:

brew upgrade https://raw.githubusercontent.com/dcos/dcos-e2e/master/dcose2e.rb

Or the latest master:

Homebrew installs the dependencies for the latest released version and so installing master may not work.

brew install --HEAD https://raw.githubusercontent.com/dcos/dcos-e2e/master/dcose2e.rb

CLI on Linux With Linuxbrew ¶

One way to install the CLI on Linux is with Linuxbrew. Install Linuxbrew, then install the latest stable version:

brew install https://raw.githubusercontent.com/dcos/dcos-e2e/master/dcose2e.rb

To upgrade from an older version, run the following command:

brew upgrade https://raw.githubusercontent.com/dcos/dcos-e2e/master/dcose2e.rb

Or the latest master:

Homebrew installs the dependencies for the latest released version and so installing master may not work.

brew install --HEAD https://raw.githubusercontent.com/dcos/dcos-e2e/master/dcose2e.rb

Library and CLI with Python ¶

If the CLI has been installed with Homebrew, you do not need to install the library to use the CLI.

Requires Python 3.5.2+. To avoid interfering with your system’s Python, we recommend using a virtualenv.

Check the Python version:

python3 --version

On Fedora, install Python development requirements:

sudo dnf install -y git python3-devel

On Ubuntu, install Python development requirements:

apt install -y gcc python3-dev

If you are not in a virtualenv, you may have to use sudo before the following command, or --user after install.

pip3 install --upgrade git+https://github.com/dcos/dcos-e2e.git@2018.10.10.0

CLI on Linux with Pre-built Packages ¶

One way to install the CLI on Linux is with a pre-built package.

# Install dcos-docker
curl --fail -L https://github.com/dcos/dcos-e2e/releases/download/2018.10.10.0/dcos-docker /usr/local/bin/dcos-docker && \
chmod +x /usr/local/bin/dcos-docker
# Install dcos-vagrant
curl --fail -L https://github.com/dcos/dcos-e2e/releases/download/2018.10.10.0/dcos-vagrant /usr/local/bin/dcos-vagrant && \
chmod +x /usr/local/bin/dcos-vagrant
# Install dcos-aws
curl --fail -L https://github.com/dcos/dcos-e2e/releases/download/2018.10.10.0/dcos-aws /usr/local/bin/dcos-aws && \
chmod +x /usr/local/bin/dcos-aws

Check that your system is ready to go ¶

Run dcos-docker doctor to confirm that your system is ready to go for the Docker Backend and the dcos-docker CLI.

Run dcos-vagrant doctor to confirm that your system is ready to go for the Vagrant Backend and the dcos-vagrant CLI.

Run dcos-aws doctor to confirm that your system is ready to go for the AWS Backend and the dcos-aws CLI.

Creating a Cluster ¶

To create a cluster you first need the link to a DC/OS release artifact.

These can be found on the releases page.

DC/OS Enterprise is also supported. Ask your sales representative for release artifacts.

Creating a cluster is possible with the dcos-aws create command. This command allows you to customize the cluster in many ways.

The command returns when the DC/OS installation process has started. To wait until DC/OS has finished installing, use the dcos-aws wait command.

To use this cluster, it is useful to find details using the dcos-aws inspect command.

DC/OS Enterprise ¶

There are multiple DC/OS Enterprise-only features available in dcos-aws create.

The only extra requirement is to give a valid license key, for DC/OS 1.11+. See dcos-aws create for details on how to provide a license key.

Ask your sales representative for DC/OS Enterprise release artifacts.

For, example, run the following to create a DC/OS Enterprise cluster in strict mode:

$ dcos-aws create $DCOS_ENTERPRISE_URL --variant enterprise \
     --license-key /path/to/license.txt \
     --security-mode strict

The command returns when the DC/OS installation process has started. To wait until DC/OS has finished installing, use the dcos-aws wait command.

See dcos-aws create for details on this command and its options.

Cluster IDs ¶

Clusters have unique IDs. Multiple commands take --cluster-id options. Specify a cluster ID in dcos-aws create, and then use it in other commands. Any command which takes a --cluster-id option defaults to using “default” if no cluster ID is given.

Running commands on Cluster Nodes ¶

It is possible to run commands on a cluster node in multiple ways. These include using dcos-aws run and ssh.

Running commands on a cluster node using dcos-aws run ¶

It is possible to run the following to run a command on an arbitrary master node.

$ dcos-aws run systemctl list-units

See dcos-aws run for more information on this command.

Running commands on a cluster node using `ssh`¶

One SSH key allows access to all nodes in the cluster. See this SSH key’s path and the IP addresses of nodes using dcos-aws inspect.

Getting on to a Cluster Node ¶

Sometimes it is useful to get onto a cluster node. To do this, you can use any of the ways of Running commands on Cluster Nodes.

For example, to use dcos-aws run to run bash to get on to an arbitrary master node:

$ dcos-aws run bash

Destroying Clusters ¶

Destroying clusters is not currently supported.

Running Integration Tests ¶

The dcos-aws run command is useful for running integration tests.

To run integration tests which are developed in the a DC/OS checkout at /path/to/dcos, you can use the following workflow:

$ dcos-aws create https://downloads.dcos.io/dcos/stable/dcos_generate_config.sh
$ dcos-aws wait
$ dcos-aws run --sync-dir /path/to/dcos/checkout pytest -k test_tls.py

There are multiple options and shortcuts for using these commands. See dcos-aws run for more information on this command.

Viewing the Web UI ¶

To view the web UI of your cluster, use the dcos-aws web command. To see the web UI URL of your cluster, use the dcos-aws inspect command.

Before viewing the UI, you may first need to configure your browser to trust your DC/OS CA, or choose to override the browser protection.

Using a Custom CA Certificate ¶

On DC/OS Enterprise clusters, it is possible to use a custom CA certificate. See the Custom CA certificate documentation for details. It is possible to use dcos-aws create to create a cluster with a custom CA certificate.

Create or obtain the necessary files:

dcos-ca-certificate.crt, dcos-ca-certificate-key.key, and dcos-ca-certificate-chain.crt.
Put the above-mentioned files into a directory, e.g. /path/to/genconf/.
Create a file containing the “extra” configuration.

dcos-aws create takes an --extra-config option. This adds the contents of the specified YAML file to a minimal DC/OS configuration.

Create a file with the following contents:
```
ca_certificate_path: genconf/dcos-ca-certificate.crt
ca_certificate_key_path: genconf/dcos-ca-certificate-key.key
ca_certificate_chain_path: genconf/dcos-ca-certificate-chain.crt
```

Create a cluster.

dcos-aws create \
    $DCOS_ENTERPRISE_URL \
    --variant enterprise \
    --genconf-dir /path/to/genconf/ \
    --copy-to-master /path/to/genconf/dcos-ca-certificate-key.key:/var/lib/dcos/pki/tls/CA/private/custom_ca.key \
    --license-key /path/to/license.txt \
    --extra-config config.yml

Verify that everything has worked.

See Verify installation for steps to verify that the DC/OS Enterprise cluster was installed properly with the custom CA certificate.

CLI Reference ¶

dcos-aws ¶

Manage DC/OS clusters on AWS.

dcos-aws [OPTIONS] COMMAND [ARGS]...

Options

--version¶: Show the version and exit.

Commands

create

doctor

inspect

list

run

sync

wait

web

dcos-aws create ¶

Create a DC/OS cluster.

DC/OS Enterprise

DC/OS Enterprise clusters require different configuration variables to DC/OS OSS. For example, enterprise clusters require the following configuration parameters:

superuser_username, superuser_password_hash, fault_domain_enabled, license_key_contents

These can all be set in --extra-config. However, some defaults are provided for all but the license key.

The default superuser username is admin. The default superuser password is admin. The default fault_domain_enabled is false.

license_key_contents must be set for DC/OS Enterprise 1.11 and above. This is set to one of the following, in order:

* The license_key_contents set in --extra-config. * The contents of the path given with --license-key. * The contents of the path set in the DCOS_LICENSE_KEY_PATH environment variable.

If none of these are set, license_key_contents is not given.

dcos-aws create [OPTIONS] ARTIFACT_URL

Options

--custom-tag <custom_tag>¶: Add tags to EC2 instances in the format “<TAG_KEY>:<TAG_VALUE>”.

--variant <variant>¶: Choose the DC/OS variant. If the variant does not match the variant of the given artifact URL, an error will occur. [required]

--masters <masters>¶: The number of master nodes. [default: 1]

--agents <agents>¶: The number of agent nodes. [default: 1]

--extra-config <extra_config>¶: The path to a file including DC/OS configuration YAML. The contents of this file will be added to add to a default configuration.

--public-agents <public_agents>¶: The number of public agent nodes. [default: 1]

--aws-region <aws_region>¶: The AWS region to use. [default: us-west-2]

--linux-distribution <linux_distribution>¶: The Linux distribution to use on the nodes. [default: centos-7]

--workspace-dir <workspace_dir>¶: Creating a cluster can use approximately 2 GB of temporary storage. Set this option to use a custom “workspace” for this temporary storage. See https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir for details on the temporary directory location if this option is not set.

--license-key <license_key>¶: This is ignored if using open source DC/OS. If using DC/OS Enterprise, this defaults to the value of the DCOS_LICENSE_KEY_PATH environment variable.

--genconf-dir <genconf_dir>¶: Path to a directory that contains additional files for the DC/OS installer. All files from this directory will be copied to the “genconf” directory before running the DC/OS installer.

--security-mode <security_mode>¶: The security mode to use for a DC/OS Enterprise cluster. This overrides any security mode set in --extra-config.

--copy-to-master <copy_to_master>¶: Files to copy to master nodes before installing DC/OS. This option can be given multiple times. Each option should be in the format /absolute/local/path:/remote/path.

-v, --verbose¶: Use verbose output. Use this option multiple times for more verbose output.

-c, --cluster-id <cluster_id>¶: A unique identifier for the cluster. Use the value “default” to use this cluster for other commands without specifying –cluster-id.

--enable-selinux-enforcing¶: With this flag set, SELinux is set to enforcing before DC/OS is installed on the cluster.

Arguments

ARTIFACT_URL¶: Required argument

Environment variables

DCOS_LICENSE_KEY_PATH: Provide a default for --license-key

dcos-aws doctor ¶

Diagnose common issues which stop this CLI from working correctly.

dcos-aws doctor [OPTIONS]

Options

-v, --verbose¶: Use verbose output. Use this option multiple times for more verbose output.

dcos-aws list ¶

List all clusters.

dcos-aws list [OPTIONS]

Options

--aws-region <aws_region>¶: The AWS region to use. [default: us-west-2]

dcos-aws run ¶

Run an arbitrary command on a node.

This command sets up the environment so that pytest can be run.

For example, run dcos-aws run --cluster-id 1231599 pytest -k test_tls.py.

Or, with sync: dcos-aws run --sync-dir . --cluster-id 1231599 pytest -k test_tls.py.

To use special characters such as single quotes in your command, wrap the whole command in double quotes.

dcos-aws run [OPTIONS] NODE_ARGS...

Options

-c, --cluster-id <cluster_id>¶: The ID of the cluster to use. [default: default]

--dcos-login-uname <dcos_login_uname>¶: The username to set the DCOS_LOGIN_UNAME environment variable to.

--dcos-login-pw <dcos_login_pw>¶: The password to set the DCOS_LOGIN_PW environment variable to.

--sync-dir <sync_dir>¶: The path to a DC/OS checkout. Part of this checkout will be synced to all master nodes before the command is run.

--no-test-env¶: With this flag set, no environment variables are set and the command is run in the home directory.

--env <env>¶: Set environment variables in the format “<KEY>=<VALUE>”

--aws-region <aws_region>¶: The AWS region to use. [default: us-west-2]

-v, --verbose¶: Use verbose output. Use this option multiple times for more verbose output.

--node <node>¶: A reference to a particular node to run the command on. This can be one of: The node’s public IP address, The node’s private IP address, the node’s EC2 instance ID, a reference in the format “<role>_<number>”. These details be seen with dcos-aws inspect.

Arguments

NODE_ARGS¶: Required argument(s)

dcos-aws inspect ¶

Show cluster details.

dcos-aws inspect [OPTIONS]

Options

-c, --cluster-id <cluster_id>¶: The ID of the cluster to use. [default: default]

--aws-region <aws_region>¶: The AWS region to use. [default: us-west-2]

-v, --verbose¶: Use verbose output. Use this option multiple times for more verbose output.

dcos-aws sync ¶

Sync files from a DC/OS checkout to master nodes.

This syncs integration test files and bootstrap files.

DCOS_CHECKOUT_DIR should be set to the path of clone of an open source DC/OS or DC/OS Enterprise repository.

By default the DCOS_CHECKOUT_DIR argument is set to the value of the DCOS_CHECKOUT_DIR environment variable.

If no DCOS_CHECKOUT_DIR is given, the current working directory is used.

dcos-aws sync [OPTIONS] [DCOS_CHECKOUT_DIR]

Options

-c, --cluster-id <cluster_id>¶: The ID of the cluster to use. [default: default]

--aws-region <aws_region>¶: The AWS region to use. [default: us-west-2]

-v, --verbose¶: Use verbose output. Use this option multiple times for more verbose output.

Arguments

DCOS_CHECKOUT_DIR¶: Optional argument

Environment variables

DCOS_CHECKOUT_DIR: Provide a default for DCOS_CHECKOUT_DIR

dcos-aws wait ¶

Wait for DC/OS to start.

dcos-aws wait [OPTIONS]

Options

-c, --cluster-id <cluster_id>¶: The ID of the cluster to use. [default: default]

--superuser-username <superuser_username>¶: The superuser username is needed only on DC/OS Enterprise clusters. By default, on a DC/OS Enterprise cluster, admin is used.

--superuser-password <superuser_password>¶: The superuser password is needed only on DC/OS Enterprise clusters. By default, on a DC/OS Enterprise cluster, admin is used.

-v, --verbose¶: Use verbose output. Use this option multiple times for more verbose output.

--aws-region <aws_region>¶: The AWS region to use. [default: us-west-2]

dcos-aws web ¶

Open the browser at the web UI.

Note that the web UI may not be available at first. Consider using dcos-aws wait before running this command.

dcos-aws web [OPTIONS]

Options

-c, --cluster-id <cluster_id>¶: The ID of the cluster to use. [default: default]

--aws-region <aws_region>¶: The AWS region to use. [default: us-west-2]

-v, --verbose¶: Use verbose output. Use this option multiple times for more verbose output.

dcos-aws CLI¶

`dcos-aws` CLI¶