How to properly rotate image and labels for semantic segmentation data augmentation in Tensorflow?

Question

What's a proper procedure for doing the image and label rotation for semantic segmentation in dataset augmentation using Tensorflow?

Images

I have seen the function tf.contib.image.rotate(), but this function fills empty space with zeros (from docs):

Empty space due to the rotation will be filled with zeros.

I would like to fill that empty space with a different value (maybe some constant, like the dataset mean pixel). How this can be done in Tensorflow (I know that there are options in Keras Image Preprocessing, but I need TF)?

Labels

Also, what about the labels? If I just use the same function (tf.contrib.image.rotate()), it will fill the empty space with zeros suggesting that pixel in those places belong to class with id 0 (since I have class labeled with 0). The one solution could be to put ignore label on those pixels (e.g. 255), but, again, the current function doesn't support default fill value ...