From Wikipedia: The public key consists of the modulus n and the public (or encryption) exponent e. The private key consists of the modulus n and the private (or decryption) exponent d, which must be kept secret. p, q, and φ(n) must also be kept secret because they can be used to calculate d.
This means that that you are correct. D is the most important part, but because 'd' can be found from other information (p,DP,DQ, InverseQ) they too should also be kept secret. The modulus and exponent form the public key.