The question was fairly vague, but that's because I really had no idea where to start. Every thing seemed to be working correctly.
After playing around with the code I noticed that I had an object that implemented IDisposable that I called Dispose on when I was done with it. The object also had a finalizer that called Dispose. I changed it to be more like this with a protected Dispose(bool)
.
I also removed the Excel assembly per Kenny Ker again (however just copying his code didn't work. I actually copied the interfaces straight from the excel interop assembly).
After doing that I was able to get rid of the lines I added (for marshalling the garbage collecting) above and excel now closes without a problem.