I found a solution to this problem, turns out you can just dump the rapidminer operation to an .rmp file (similiar to xml) and let the java code run rapidminer to open the .rmp to run the whole operation (R extensions operations included), here's the source http://how2-code.blogspot.com/2013/10/how-to-read-rmp-file-in-rapidminer-java.html
I hope this can be useful.