Replace your line :
code.addRCode("boxplot((1:10),main=\"1-10\")");
By this :
code.addRCode("boxplot(c(1:10),main='1-10')");
Or as in the second example (the working one), you can give your x vector from java and replace this line:
caller.runAndReturnResultOnline("plot.ts(x)");
by
caller.runAndReturnResultOnline("boxplot(x)");
PS: I dont' have java to test.