As you have not posted any code ..I can tell only logic .. U can get the selectedtab index by
var index= ($tabs.tabs('option', 'selected'));
and then you compare your index with the fetched index and if it matches then open your fancybox by
$(".fancy").fancybox().trigger('click');
and you can write your own function on sumbit/close of fancybox
$("<YOUR-SELECTOR>").fancybox({
onClosed: function() {
// your own functions
});
});