You should call Process.CloseMainWindow which performs an orderly termination of the process and closes all windows. As opposed to Process.Kill which causes abnormal termination. CloseMainWindow is preferable for applications with a user interface.
process = Process.Start(exe);//in form load set field
private void button1_Click(object sender, EventArgs e)
{
process.CloseMainWindow();
}
The remarks on MSDN reveals important information regarding the asynchronous behavior and other relevant details.
Kill forces a termination of the process, while CloseMainWindow only
requests a termination. When a process with a graphical interface is
executing, its message loop is in a wait state. The message loop
executes every time a Windows message is sent to the process by the
operating system. Calling CloseMainWindow sends a request to close to
the main window, which, in a well-formed application, closes child
windows and revokes all running message loops for the application. The
request to exit the process by calling CloseMainWindow does not force
the application to quit. The application can ask for user verification
before quitting, or it can refuse to quit. To force the application to
quit, use the Kill method. The behavior of CloseMainWindow is
identical to that of a user closing an application's main window using
the system menu. Therefore, the request to exit the process by closing
the main window does not force the application to quit immediately.
etc...