The way to go is send phantomjs special keys.
casper.then(function() {
test.comment('Send Esc key');
this.sendKeys('#contact-us-modal', casper.page.event.key.Escape);
});
Here is the list of special keys available in casper.page.event.key
Here is the sendKeys documentation link.