I found that it pops a new window instead of just being redirected,
Yep, this is what Intuit wants it to do.
Is there any way to keep the button from popping a new window and just redirect in the same tab?
Nope.
Trying to do that is a great way to get Intuit to ban your application/remove it from AppCenter.
Or, if there really isn't anything would it be possible to close the popped up window automatically and resume control of the previously used tab?
Of course. In fact, you'll have to if you want to publish on AppCenter.
Simple Javascript - window.close();
Keep in mind that this window only has to be opened once and then never again. It's not like this is going to be a huge inconvenience to your users when they'll only need to do this once in the entire history of ever using your application.