I believe the ones in VLfeat are already oriented in the dominant gradient direction.
It shows them rotated if you look here: http://www.vlfeat.org/overview/sift.html
[f,d] = vl_sift(I) ;
f is a Nx4 matrix of keypoints. N being the keypoint indexand the other 4 being, x position, y position, scale, and orientation. d is a Nx128 matrix, where N is the keypoint index, and the 128 dimensions belong to the SIFT descriptor.
If all of your images are rotated upright, it is actually beneficial to not use rotational invariance. See this paper which assume a gravity vector: https://dspace.cvut.cz/bitstream/handle/10467/9548/2009-Efficient-representation-of-local-geometry-for-large-scale-object-retrieval.pdf?sequence=1