Docker Backend¶
The Docker backend is used to spin up clusters on Docker containers, where each container is a DC/OS node.
Requirements¶
Docker¶
Docker version 17.06 or later must be installed.
Plenty of memory must be given to Docker. On Docker for Mac, this can be done from Docker > Preferences > Advanced. This backend has been tested with a four node cluster with 9 GB memory given to Docker.
IP Routing Set Up for Docker¶
On macOS, hosts cannot connect to containers IP addresses by default. This is required, for example, to access the web UI, to SSH to nodes and to use the DC/OS CLI.
Once the CLI is installed, run dcos-docker setup-mac-network to set up IP routing.
Without this, it is still possible to use some features.
In the library, specify transport
as dcos_e2e.node.Transport.DOCKER_EXEC
.
In the CLI, specify the --transport
and --skip-http-checks
options where available.
Operating System¶
This tool has been tested on macOS with Docker for Mac and on Linux.
It has also been tested on Windows on Vagrant.
Windows¶
The only supported way to use the Docker backend on Windows is using Vagrant and VirtualBox.
- Ensure Virtualization and VT-X support is enabled in your PC’s BIOS. Disable Hyper-V virtualization. See https://www.howtogeek.com/213795/how-to-enable-intel-vt-x-in-your-computers-bios-or-uefi-firmware/.
- Install VirtualBox and VirtualBox Extension Pack.
- Install Vagrant.
- Install the Vagrant plugin for persistent disks:
vagrant plugin install vagrant-persistent-storage
- Optionally install the Vagrant plugins to cache package downloads and keep guest additions updates:
vagrant plugin install vagrant-cachier
vagrant plugin install vagrant-vbguest
- Start Powershell and download the DC/OS E2E
Vagrantfile
to a directory containing a DC/OS installer file:
((New-Object System.Net.WebClient).DownloadString('https://raw.githubusercontent.com/dcos/dcos-e2e/master/vagrant/Vagrantfile')) | Set-Content -LiteralPath Vagrantfile
- By default, the
Vagrantfile
installs DC/OS E2E from the most recent release at the time it is downloaded. To use a different release, or any Git reference, set the environment variableDCOS_E2E_REF
:
$env:DCOS_E2E_REF = "master"
- Start the virtual machine and login:
vagrant up
vagrant ssh
You can now run dcos-docker CLI commands or use the Python Library.
To connect to the cluster nodes from the Windows host (e.g. to use the DC/OS web interface), in PowerShell Run as Administrator, and add the Virtual Machine as a gateway:
route add 172.17.0.0 MASK 255.255.0.0 192.168.18.2
To shutdown, logout of the virtual machine shell, and destroy the virtual machine and disk:
vagrant destroy
The route will be removed on reboot. You can manually remove the route in PowerShell Run as Administrator using:
route delete 172.17.0.0
dcos-docker doctor
¶
DC/OS E2E comes with the dcos-docker doctor command. Run this command to check your system for common causes of problems.
DC/OS Installation¶
Cluster
s created by the Docker backend only support installing DC/OS via install_dcos_from_path()
.
Node
s of Cluster
s created by the Docker backend do not distinguish between public_ip_address
and private_ip_address
.
Limitations¶
Docker does not represent a real DC/OS environment with complete accuracy. This section describes the currently known differences between the Docker backend and a real DC/OS environment.
SELinux¶
Tests inherit the host’s environment. Any tests that rely on SELinux being available require it be available on the host.
Storage¶
Docker does not support storage features expected in a real DC/OS environment.
Troubleshooting¶
Cleaning Up and Fixing “Out of Space” Errors¶
If a test is interrupted, it can leave behind containers, volumes and files. To remove these, run the following:
docker stop $(docker ps -a -q --filter="name=dcos-e2e")
docker rm --volumes $(docker ps -a -q --filter="name=dcos-e2e")
docker volume prune --force
If this repository is available, run make clean
.
macOS File Sharing¶
On macOS /tmp
is a symlink to /private/tmp
.
/tmp
is used by the harness.
Docker for Mac must be configured to allow /private
to be bind mounted into Docker containers.
This is the default.
See Docker > Preferences > File Sharing.
Clock sync errors¶
On various platforms, the clock can get out of sync between the host machine and Docker containers.
This is particularly problematic if using check_time: true
in the DC/OS configuration.
To work around this, run docker run --rm --privileged alpine hwclock -s
.
Reference¶
-
class
dcos_e2e.backends.
Docker
(workspace_dir=None, custom_container_mounts=None, custom_master_mounts=None, custom_agent_mounts=None, custom_public_agent_mounts=None, linux_distribution=<Distribution.CENTOS_7: 1>, docker_version=<DockerVersion.v1_13_1: 2>, storage_driver=None, docker_container_labels=None, docker_master_labels=None, docker_agent_labels=None, docker_public_agent_labels=None, transport=<Transport.SSH: 1>, network=None)¶ Create a configuration for a Docker cluster backend.
Parameters: - workspace_dir (
Optional
[Path
]) – The directory in which large temporary files will be created. These files will be deleted at the end of a test run. This is equivalent to dir intempfile.mkstemp()
. - custom_container_mounts (
Optional
[List
[Mount
]]) – Custom mounts add to all node containers. See mounts in Containers.run. - custom_master_mounts (
Optional
[List
[Mount
]]) – Custom mounts add to master node containers. See mounts in Containers.run. - custom_agent_mounts (
Optional
[List
[Mount
]]) – Custom mounts add to agent node containers. See mounts in Containers.run. - custom_public_agent_mounts (
Optional
[List
[Mount
]]) – Custom mounts add to public agent node containers. See mounts in Containers.run. - linux_distribution (
Distribution
) – The Linux distribution to boot DC/OS on. - docker_version (
DockerVersion
) – The Docker version to install on the cluster nodes. - storage_driver (
Optional
[DockerStorageDriver
]) – The storage driver to use for Docker on the cluster nodes. By default, this is the host’s storage driver. If this is not one ofaufs
,overlay
oroverlay2
,aufs
is used. - docker_container_labels (
Optional
[Dict
[str
,str
]]) – Docker labels to add to the cluster node containers. Akin to the dictionary option in Containers.run. - docker_master_labels (
Optional
[Dict
[str
,str
]]) – Docker labels to add to the cluster master node containers. Akin to the dictionary option in Containers.run. - docker_agent_labels (
Optional
[Dict
[str
,str
]]) – Docker labels to add to the cluster agent node containers. Akin to the dictionary option in Containers.run. - docker_public_agent_labels (
Optional
[Dict
[str
,str
]]) – Docker labels to add to the cluster public agent node containers. Akin to the dictionary option in Containers.run. - transport (
Transport
) – The transport to use for communicating with nodes. - network (
Optional
[Network
]) – The Docker network containers will be connected to. If no network is specified thedocker0
bridge network is used. It may not be possible to SSH to containers on a custom network on macOS. Therefore, it is recommended that you use this in conjunction with thetransport
parameter.
-
workspace_dir
¶ The directory in which large temporary files will be created. These files will be deleted at the end of a test run.
-
custom_container_mounts
¶ Custom mounts add to all node containers. See mounts in Containers.run.
-
custom_master_mounts
¶ Custom mounts add to master node containers. See mounts in Containers.run.
-
custom_agent_mounts
¶ Custom mounts add to agent node containers. See mounts in Containers.run.
-
custom_public_agent_mounts
¶ Custom mounts add to public agent node containers. See mounts in Containers.run.
-
linux_distribution
¶ The Linux distribution to boot DC/OS on.
-
docker_version
¶ The Docker version to install on the cluster nodes.
-
docker_storage_driver
¶ The storage driver to use for Docker on the cluster nodes.
-
docker_container_labels
¶ Docker labels to add to the cluster node containers. Akin to the dictionary option in Containers.run.
-
docker_master_labels
¶ Docker labels to add to the cluster master node containers. Akin to the dictionary option in Containers.run.
-
docker_agent_labels
¶ Docker labels to add to the cluster agent node containers. Akin to the dictionary option in Containers.run.
-
docker_public_agent_labels
¶ Docker labels to add to the cluster public agent node containers. Akin to the dictionary option in Containers.run.
-
transport
¶ The transport to use for communicating with nodes.
-
network
¶ The Docker network containers will be connected to. If no network is specified the
docker0
bridge network is used. It may not be possible to SSH to containers on a custom network on macOS. Therefore, it is recommended that you use this in conjunction with thetransport
parameter.
- workspace_dir (