Try
cv::Rect roi(padSize, padSize, rpad.cols-2*padSize, rpad.rows-2*padSize);
cv::Mat result = rpad(roi);
And depending on whether you want continuous memory, you can choose to directly use result (discontinuous, usually okay for most OpenCV functions) or copy it to back to rpad (continuous)