Securing Docker Containers on AWS

In 2015 the Docker team set out to consolidate all the information around best practices for securing docker containers.

You can find references to the CIS Docker Community Edition Benchmark version 1.1.0 and Docker's own white paper Introduction to Docker Security on the topic here .

The links above get you more directly to the assets. Just an FYI, the CIS benchmark will cost you an email address to access the download.

On most projects at NearForm we are deploying our solutions within Docker containers.

There are tasks that are repeated on each project to secure and harden off those deployments, and we built this packer template to produce a quick and easy way for you to spin up an AWS AMI that passes the Docker-Bench-Security script.

The Docker-Bench-Security repo is a work product of the above-mentioned consolidation efforts by the Docker team.

To accomplish the building of this AMI we use Packer , which is an easy way to automate the creation of your images. It supports multiple providers (i.e. AWS, Digital Ocean) and allows you to document in a repo how your images were built and modifications that were done to them over time.

The work to get this AMI passing the Docker-Bench-Security was not a small task, but a critical task of any development organization.

Minimizing the attack surface of an application we deploy into the wild will always return on investment.

Whether that return comes from keeping an active attacker from escalating privileges and damaging our craft or simply letting you sleep at night knowing that you've done everything you can to secure your environments, give it a spin and feel free to raise issues or PRs if you find ways to improve upon the work.

Just give me an AMI I can work with

To get things started we are using Packer to create an ebs-backed AMI with the underlying OS being Ubuntu 17.0.4.

We are also installing Docker CE for Ubuntu with the supporting packages suggested by the Docker Team.

Out of the box, our base Ubuntu AMI is failing the following tests in the Docker-Bench-Security .

We'll go through each and document how we passed each test.

To pass 1.1 we had to do a few tasks that included modifying our underlying packer template and then scripting a few changes for the AMI build.

The volume that is defined in the ami_block_device_mappings will be setup and used as the mount for all things Docker. (i.e. images, containers).

Few notes on this block device: - it's currently set for 1GB in size, which is not ideal for anything but testing. (i.e. "volume_size": 1 ) - "device_name" : "/dev/sdf" gets renamed hence the different name in our ./scripts/volumn.sh From my research this appeared to be because of the specific AMI we are using, but to dig deeper on the warning from AWS and reasoning behind them read more The actual work of mounting and configuring the disk for operation is completed in ./scripts/volumn.sh .

The underlying host of the AMI we create IS NOT hardened and the Docker-Bench-Security DOES NOT verify the host's settings.

You could easily switch the AMI ID that we are using in our example with one from CIS Benchmarks and start fresh with a hardened host and a secure docker implementation.

These tests are passed with the installation and configuration of auditd , which is all done within auditd.sh .

With our settings we are watching for writes and changes to the attributes of each item. more info Each test is listed above the setting for reference.

We accomplish passing these test with the creation of /etc/docker/daemon.json from ./files/daemon.json .

This warning seems to contradict 5.29, which we implment a solution for alter on in this post.

Since we created our new parition with ext4 we are able to use the preferred storage-driver which is overlay2. - Docker CE on Ubuntu storage drivers - supported backing filesystems

"The best way to prevent privilege-escalation attacks from within a container is to configure your containers applications to run as unprivileged users." source Using default means our user will be dockremap versus something that may make more sense in your environment.

We do not implement changes (i.e we still get a WARN result) for this since it requires generation of keys and certificates that will be unique to your environment.

You can read more on the details of how to configure the docker daemon and how to protect the docker socket

We are writing everything to syslog, but you could easily configure aws-logs or another end point that you actively work with for logging.

This is possibly a legacy configuation since support *"for the v1 protocol to the public registry was removed in 1.13"*.

This setting allows your containers to continue running when there are issues that cause the docker daemon to be down. more info

This warning is looking for you to run your processes within the container as seperate users.

With userns-remap enabled for our containers this seems a bit excessive since the container is restricted to a lower privileged user.

This could be a wrong assumption, but restricting users that run your process inside the container could have other adverse effects. This should be evaluated and configured based on your applications needs.

This test was passed by modifying the Dockerfile to include adding a new user and group.

You can confirm your settings are in place with the following.

We take care of this by setting the environment variable DOCKER_CONTENT_TRUST for all users in ./scripts/docker-setup.sh .

Note that if you build an image locally there are known issues with Docker Content Trust disallowing your action with an authorization failure (i.e. 401).

You can use the --disable-content-trust flag to bypass the setting we created in our docker setup script

Docker Content Trust has no concept of trusting something built locally #25852

Adding the following lines to the Dockerfile gets us past this test. This checks the url every 20s and fails after 3s.

It appears the node:8 image has a few items that docker-bench-security does not like.

Running docker history on the image we build with our example will show that those commands are getting into our images based on FROM node:8

By default, a docker container has no resource constraints.

You can set specific runtime flags for both Memory and CPU based on your applications needs. more info I had no success with the proposed flags and passing the tests that docker-bench-security runs for them.

This can be accomplished by adding the flag --restart to your docker run command as below:

This can be accomplished by adding the flag --security-opt to your docker run command

I've yet to find a solution that works and passes the test.

The runtime flag --pids-limit does not appear to have any effect on the configuration of a container.

If you are interested in using a bridge other than docker0 you can build your own bridge

If you want to use a different region in AWS, change the aws_region and find replace the ubuntu_source_ami with the AMI ID that has an instance type of hvm-ssd from Ubuntu Your File should look something like this:

Our example template uses packer's amazon-ebs builder.

We use a builder and two provisioners in our template to accomplish the following:

Once packer has finished building your AMI, you can log into the AWS Console and launch an instance from the AMI.

You will only need to configure network related items (i.e. subnet) and the rest you should be able to take the defaults.

Once your instance is up and running ssh into it and you should find docker-bench-security in the home directory of the ubuntu user.

Do the following to run and verify the AMI passes the test we've documented above.

*note*: sudo is used in step 2 due to checks against files and directories that require elevated privileges.

If you would like to build a docker image and run a container based on that image you can include ./files/app/ in your AMI build, which will give you a simple hapi app to run the Docker-Bench-Security tests against.

To include these files add the following to the provisioners section of the template