I don't know if it's your case but this behavior can happen due to some attributes in the CSS file that is loaded by the htmlArea: RTE.default.contentCSS
I'm not able to tell you in which versions of IE this can happen (I can reproduce it in IE 7) and I don't know all the attributes that might cause this but I know that for example min-height
might trigger this.
So start with debugging the CSS by removing atttributes from it. Make sure the browser loads the new version of the file after each change and before the content with the RTE is opened again! (open the file in a new tab to check the content that the browser uses)