How can I improve the quality of this python remote service code?

Question

I wrote several small Python scripts to start clusters inside cloud infrastructures with distributed filesystems. Now, I integrated the code into a single command-line application but the code quality is very bad.

http://code.google.com/p/diffuser/source/browse/trunk/diffuser.py

The application needs to send lots of commands via SSH with paramiko. Each command that is send via SSH needs three lines of code:

stdin, stdout, stderr = client.exec_command("<command>")
print stdout.readlines()
print stderr.readlines()

How can I improve the code quality?

Answer 1

Commands are executed in a shell, so you can use regular shell syntax to combine them together. As a first step, I'd issue a batch of commands in a single exec_command:

  stdin, stdout, stderr = client.exec_command(
           "sudo hostname;"
           "sudo apt-get update;"
           "sudo apt-get -y install nfs-common nfs-kernel-server;"
           "echo y | sudo mkfs.ext3 /dev/sdc;"
           "sudo mkdir /mnt/export;"
           "sudo mount /dev/sdc /mnt/export/;"
           "sudo chmod o+wx /etc/exports;")
  print stdout.readlines()
  print stderr.readlines()

In addition, I consider it unnecessary to start a fresh sudo for each of them. So I'd rather write

  stdin, stdout, stderr = client.exec_command(
           "sudo /bin/sh -c '"
           "hostname;"
           "apt-get update;"
           "apt-get -y install nfs-common nfs-kernel-server;"
           "echo y | mkfs.ext3 /dev/sdc;"
           "mkdir /mnt/export;"
           "mount /dev/sdc /mnt/export/;"
           "chmod o+wx /etc/exports;"
           "'")
  print stdout.readlines()
  print stderr.readlines()

Answer 2

The first thing I would do is improve the readability of what is going on by adding some functions, as it stands now it's one big script with too many levels of indentation and redirection, which makes it difficult to read.

To begin with, now that you have a command-line application, you should first ensure that this is how it's being run. You should also make filesystem selection and anything else that needs to be chosen at runtime a commandline switch. If the user enters something incorrect, print usage and exit. Maybe something more like this:

if __name__ == '__main__':
    filesystem, other_thing = parse_args(sys.argv)
    config = read_config()
    if filesystem in valid_filesystems and valid_thing(other_thing):
        start_client(config, filesystem)
        start_server(whatever)
    else:
        print_usage()
        sys.exit(0)

and then add your top level control flow to start_client / start server. You may also want to create a GenericClient class and a GenericServer class, which you inherit from and modify depending on the filesystem selected.

In general, I'd read up a bit on Object Orientation in python. I've found one guide here but there may be some better ones that others can suggest.