I had success with the answer above, but then had trouble dismissing the modal in the case that the 'close' button was pressed. The solution was to add the code
[Export ("windowWillClose:")]
public void WindowWillClose(NSNotification notification)
{
Console.WriteLine("windowWillClose:");
NSApplication.SharedApplication.StopModal ();
}
to the window controller, then set the window controller to be the window's delegate by, in Interface Builder, right-click the window and drag a line from "delegate" to the "file's owner" block.