I contacted Paypal support and this is what they had to say:
I checked your current IPN settings for your account and it appears IPN is set to 'Off', not 'Disabled'. This is a very important difference, as IPN being disabled prevents IPN messages from being sent at all. If IPN is simply turned off, PayPal will only send IPN messages if a notify_url is specified in the payment request. For Adaptive Payments, the notify_URL is the only way to trigger IPN messages, so IPN can be in the 'Off' setting and worked correctly, but not disabled.
This was received after I randomly fixed the problem myself. The way I got it to work was simply playing around with my account settings. I enabled IPN in my account and then disabled it, after this all my IPN requests came flooding in to my IPN URL set in my Adaptive Payments API call.