HTML-Entity escaping to prevent XSS

Question

I have some user input. Within my code, I ensure that the following symbols are escaped:

& -> & 
< -> &lt; 
> -> &gt;

OWASP states that there are more chars to be escaped.

For attributes, I do another kind of escaping:

& -> & 
" -> "

This ensures that all attributes are enclosed by ". This makes me sure about my html-attributes, but not about HTML itself.

I wonder if my escaping is sufficient. I've read this post, but I'm still not sure about my concern.

(JavaScripts are escaped with the OWASP-Library)