Firstly I think recommendations in Mahout come in two parts: non-distributed algorithms (from Taste) and distributed hadoop jobs (ParallelAls and Item-based Collaborative Filtering). The RMSEEvaluator is in the non-distributed part, don't think it works with hadoop jobs.
For evaluating a hadoop recommendation job in Mahout, have a look at
org.apache.mahout.cf.taste.hadoop.als.FactorizationEvaluator
You can also write some scripts and download the recommendation results to local filesystem for evaluation.