The same way as you set username and password, you can set host, port and encryption:
$transport = $fromYourMailerVariable->getTransport();
$transport
->setUserName(...)
->setPassword(...)
->setHost(...)
->setPort(...)
->setEncryption(...);
Or just create different mailers (with different transports):
$transport1 = Swift_SmtpTransport::newInstance('localhost');
$transport2 = Swift_SmtpTransport::newInstance('smtp.domain.com', 666)
->setUsername('user')
->setPassword('pass');
$transport3 = Swift_SendmailTransport::newInstance('/usr/sbin/exim -bs');
$transport4 = Swift_MailTransport::newInstance();