(Since it solves the problem of OP, I am making my comment an answer)
You can use copyMakeBorder()
function in OpenCV to pad elements in whatever directions you like and it supports several types of padding. See the docs : http://docs.opencv.org/modules/imgproc/doc/filtering.html?highlight=makeborder#copymakeborder
BORDER_CONSTANT
will add a constant value.
BORDER_REPLICATE
will copy the border elements as such, and I think that is what you need in your problem.