Why use heap over red-black tree?

Question

Heap supports insert operation in $O(\log n)$ time. And while heap supports remove min/max in $O(\log n)$ time, to remove any element (non min/max) heap takes $O(n)$ time.

However, red-black tree supports insert/remove both in $O(\log n)$ time. We can just remove the first/last element in a red-black tree to remove min/max in $O(\log n)$ time.

I do understand that heaps are used specifically to remove the min/max, but it seems like red-black trees can do what heaps can do but just faster/equal.

Is there any distinctive advantage to using heaps over red-black trees?