Run a complex series of commands in the same Docker container

Question

I'm trying to automate the following loop with Docker: spawn a container, do some work inside of it (more than one single command), get some data out of the container.

Something along the lines of:

for ( i = 0; i < 10; i++ )
    spawn a container
    wget revision-i
    do something with it and store results in results.txt

According to the documentation I should go with:

for ( ... )
    docker run <image> <long; list; of; instructions; separated; by; semicolon>

Unfortunately, this approach is not attractive nor maintanable as the list of instructions grows in complexity.

Wrapping the instructions in a script as in docker run <image> /bin/bash script.sh doesn't work either since I want to spawn a new container for every iteration of the loop.

To sum up:

1. Is there any sensible way to run a complex series of commands as described above inside the same container?

2. Once some data are saved inside a container in, say, /home/results.txt, and the container returns, how do I get results.txt? The only way I can think of is to commit the container and tar the file out of the new image. Is there a more efficient way to do it?

Bonus: should I use vanilla LXC instead? I don't have any experience with it though so I'm not sure. Thanks.

Answer 1

I eventually came up with a solution that works for me and greatly improved my Docker experience.

Long story short: I used a combination of Fabric and a container running sshd.

Details:

The idea is to spawn container(s) with sshd running using Fabric's local, and run commands on the containers using Fabric's run.

To give a (Python) example, you might have a Container class with:

1) a method to locally spawn a new container with sshd up and running, e.g.

local('docker run -d -p 22 your/image /usr/sbin/sshd -D')

2) set the env parameters needed by Fabric to connect to the running container - check Fabric's tutorial for more on this

3) write your methods to run everything you want in the container exploiting Fabric's run, e.g.

run('uname -on')

Oh, and if you like Ruby better you can achieve the same using Capistrano.

Thanks to @qkrijger (+1'd) for putting me on the right track :)

Answer 2

On question 2.

I don't know if this is the best way, but you could install SSH on you image and use that. For more information on this, you can check out this page from the documentation.

Answer 3

You post 2 questions in one. Maybe you should put 2. in a different post. I will consider 1. here.

It is unclear to me whether you want to spawn a new container for every iteration (as you say first) or if you want to "run a complex series of commands as described above inside the same container?" as you say later.

If you want to spawn multiple containers I would expect you to have a script on your machine handling that. If you need to pass an argument to your container (like i): there is being work done on passing arguments currently. See https://github.com/dotcloud/docker/pull/1015 (and https://github.com/dotcloud/docker/pull/1015/files for documentation change which is not online yet).