I'm not sure to understand exactly what you want to do, but these may help you, I did this with a chessboard:
std::vector<cv::Point2f> not_a_rect_shape; // In this vector you should save the x,y coordinates of your 3D rect
// the not_a_rect_shapecoordinates are from you source frame
// Define the destination image
quad = cv::Mat::zeros(300, 220, CV_8UC3); // the size is important, it depends on your application, I have to say that 300x220 isn't always the right one. you have to give more infomations!!
// these will be the parallel plane vector of point
quad_pts.push_back(cv::Point2f(0, 0));
quad_pts.push_back(cv::Point2f(quad.cols, 0));
quad_pts.push_back(cv::Point2f(quad.cols, quad.rows));
quad_pts.push_back(cv::Point2f(0,quad.rows));
transformationMatrix= cv::getPerspectiveTransform(not_a_rect_shape, quad_pts);// this is you transformation matirx
cv::warpPerspective(src, quad, transformationMatrix,quad.size(),CV_INTER_LINEAR,cv::BORDER_ISOLATED); // the flags could change
now quad frame should be that what you're looking for! I hope it helps