The solution I went with for this problem was to use the ProcessingEnvironment to cast the resultant Class objects to TypeMirrors, in the cases when I got classes instead of TypeMirrors. This seems to work fairly well.
AnnotationValue annValue = entry.getValue();
if (annValue instanceof TypeMirror) {
return (TypeMirror) annValue;
}
else {
String valString = annValue.getValue().toString();
TypeElement elem = processingEnv.getElementUtils().getTypeElement(valString);
return elem.asType();
}