how do I clean up my docker host machine

Question 1

It can also be helpful to remove "dangling" images

docker rmi $(docker images -f "dangling=true" -q)

Question 2

Here's how I periodically purge my docker host:

Kill running containers:

docker kill $(docker ps -qa)

Delete all containers (and their associated volumes):

docker rm -v $(docker ps -qa)

Remove all images:

docker rmi $(docker images -q)

Update

Delete only the containers that are not running. Parse the "ps" output for the "Exited" string:

docker ps -a | awk '/Exited/ {print $1}' | xargs docker rm -v

Not perfect... Don't give your container the name "Exited" :-)

Update

Docker 1.9 has a new volume command that can be used to purge old volumes

docker volume rm $(docker volume ls -qf dangling=true)

Update

Latest community edition of docker has a new "system prune" command

docker system prune --volumes

This purged unused networks, kill stopped containers, dangling images and any unused volumes.

Question 3

I would also like to contribute to this with some commands that were added to version 1.13.0:

$ docker system prune
$ docker container prune
$ docker image prune
$ docker volume prune
$ docker network prune

see changelog: 1.13.0 (2017-01-18)

Add new docker system command with df and prune subcommands for system resource management, as well as docker {container,image,volume,network} prune subcommands #26108 #27525 / #27525

Question 4

I'm using docker-machine with VirtualBox and after deleting all containers and all images, the docker VirtualBox image is still consuming many gigabytes of disk space.

To also clean up the disk space, it helps to delete and re-create the docker machine. E.g.:

docker-machine rm default
docker-machine create --driver virtualbox default

Question 5

Update: There are built-in filters in the docker cli that let one properly display containers that meet certain criteria. These bash functions are taken from one of the core maintainers' dotfiles:

dcleanup(){
    docker rm -v $(docker ps --filter status=exited -q 2>/dev/null) 2>/dev/null
    docker rmi $(docker images --filter dangling=true -q 2>/dev/null) 2>/dev/null
}
del_stopped(){
    local name=$1
    local state=$(docker inspect --format "{{.State.Running}}" $name 2>/dev/null)

    if [[ "$state" == "false" ]]; then
        docker rm $name
    fi
}

Original Answer

A helper script I've created for my own use:

#!/bin/bash

# options:
# remove stopped containers and untagged images
#     $ dkcleanup
# remove all stopped|running containers and untagged images
#     $ dkcleanup --reset
# remove containers|images|tags matching {repository|image|repository\image|tag|image:tag}
# pattern and untagged images
#     $ dkcleanup --purge {image}
# everything
#     $ dkcleanup --nuclear

if [ "$1" == "--reset" ]; then
    # Remove all containers regardless of state
    docker rm -f $(docker ps -a -q) 2>/dev/null || echo "No more containers to remove."
elif [ "$1" == "--purge" ]; then
    # Attempt to remove running containers that are using the images we're trying to purge first.
    (docker rm -f $(docker ps -a | grep "$2/\|/$2 \| $2 \|:$2\|$2-\|$2:\|$2_" | awk '{print $1}') 2>/dev/null || echo "No containers using the \"$2\" image, continuing purge.") &&\
    # Remove all images matching arg given after "--purge"
    docker rmi $(docker images | grep "$2/\|/$2 \| $2 \|$2 \|$2-\|$2_" | awk '{print $3}') 2>/dev/null || echo "No images matching \"$2\" to purge."
else
    # This alternate only removes "stopped" containers
    docker rm -f $(docker ps -a | grep "Exited" | awk '{print $1}') 2>/dev/null || echo "No stopped containers to remove."
fi

if [ "$1" == "--nuclear" ]; then
    docker rm -f $(docker ps -a -q) 2>/dev/null || echo "No more containers to remove."
    docker rmi $(docker images -q) 2>/dev/null || echo "No more images to remove."
else
    # Always remove untagged images
    docker rmi $(docker images | grep "<none>" | awk '{print $3}') 2>/dev/null || echo "No untagged images to delete."
fi

exit 0

source

To your questions:

how does one properly cleanup?

no official way yet, just helper scripts and functions like the above.

as I was manually deleting images more started to arrive. huh?

you might have been deleting images that were built on top of others that became "untagged" when you tried to delete them.

how much disk space should I really allocate to the host?

depends on the types of images you plan to use. Know that running a 500 mb image multiple times doesn't use (500mb X number of containers) space. The containers reuse the same image and just add whatever they change when running on top. So think from an image storing perspective, not a container runtime one regarding storage.

will running daemons really restart after the next reboot?

By default, they are stopped when the host reboots. You need to run with docker run --restart=True to automatically start up again when the host reboots.

Question 6

Sometimes you wont have Status, it'll just be blank.

here is my version:

docker rm -f $(docker ps -a | env -i grep -v Up | tail -n+2 | cut -d ' ' -f 1)