XmlWriter is probably the quickest, but this is like writing assembly when you could be writing in C#. I'd stay away in terms of readability.
I have no love of XmlTextWriter, in my opinion you may as well stick with XmlWriter.
XmlDocument has a lot of overhead. It's just too slow for almost anything you may want to do.
I don't trust or like XmlSerializer. It always seems to fragile to me. A future developer can unwittingly screw things up just by changing a few class definitions.
XDocument is pretty damn good. I've tried to do speed tests of XmlReader/XmlWriter against XDocument before and the two are often so close in performance, that I'd almost recommend XDocument in all cases. I say almost because XmlReader/XmlWriter is still the technically fastest way to read/write xml I've found.