Small alpha
When w*sin(alpha) < h*(1+cos(alpha))
(i.e., before the vertices of the new rectangle meet the vertices of the old one for the first time), the area of the intersection is the area of the original rectangle (w * h
) minus 4 triangles (2 pairs of identical ones). Let the bigger triangle have hypotenuse a
and the smaller hypotenuse b
, then the area is
A = w * h - a*a*cos(alpha)*sin(alpha) - b*b*cos(alpha)*sin(alpha)
The sides of the original rectangle satisfy a system of equations:
a + a * cos(alpha) + b * sin(alpha) = w
a * sin(alpha) + b + b * cos(alpha) = h
Using the half-angle formulas,
a * cos(alpha/2) + b * sin(alpha/2) = w/(2*cos(alpha/2))
a * sin(alpha/2) + b * cos(alpha/2) = h/(2*cos(alpha/2))
thus (the matrix on the LHS is a rotation!)
a^2 + b^2 = (w^2 + h^2) / (2*cos(alpha/2))^2
and
A = h * w - (w^2 + h^2) * cos(alpha)* sin(alpha) / (2*cos(alpha/2))^2
(this can be simplified further a little bit)
Bigger alpha
When alpha
is bigger (but still alpha<pi/2
), the intersection is a parallelogram (actually, a rhombus) whose 2 altitudes are h
and 4 sides h/sin(alpha)
and the area is, thus, h*h/sin(alpha)
(yes, it does not depend on w
!)
Other alpha
Use symmetry to reduce alpha to [0;pi/2]
and use one of the two cases above.