MatisBT by default disallows framing, and this is not configurable. You can patch your installation by commenting out the following line in core/http_api.php:
header( 'X-Frame-Options: DENY' );
The bug tracking making this setting configurable is 12165: Allow mantis to be loaded in an iframe.