SDL UI Menu Button - Mouse Over

Question

I am creating a menu system for my game engine and want to know how to be able to detect when the mouse is over a button. This is simple enough to do when the button is a square, rectangle or circle but I was wondering how to handle irregular shaped buttons.

Is this possible and if it is, does the complexity mean that it is better to simply use a bounding area (square or circle)?

Answer 1

Make a bitmask out of the texture or surface data. Decide on a rule; for example where the image is 100% transparent or a certain color the bitmask pixel is set to 0 otherwise set it to 1. Do the same for your cursor. When you check for collision simply check if the bitmask bits set to 1 overlap.

Answer 2

First what comes to my mind is to use mathematical functions. If you know the equation of the curve you can calculate if the point is under or over it by simply checking if right side of the equation is greater or less than the "y".

So if you have simple y = x*x and want to check point (2,1), you substitute it and check:

y = 2
x = 1*1 = 1

y > 1, point is over the curve. For opposite situation, taking the point (1,2), we get:

y = 1
x = 2*2 = 4

y < x, point is under the curve.