This hang was caused by exiting the application before the WorkflowApplication was finished. We were exiting after receiving the "Completed" event but, we needed to wait for Completed and Unloaded.
It was difficult to track down because the hang wasn't in our code and it wasn't in Workflow Foundation code. It was a loop, waiting for a counter to decrement to zero but, nothing was going to decrement the counter. We had Microsoft looking at process dumps, they could see why it was hung but could not find what sequence of events led to the hang because it would only hang 1 out of 1000 times and when we inserted instrumentation, it would not hang.
We finally found the cause by putting a delay just before the return and we realized that we were getting the Unload event that we didn't see without the delay.