I believe you really need no maven, this should work.
Your own class doesn't get found and I think it's a problem of your classpath. As they're usually more problem with nested classes try simply
java -cp caliper-1.0-beta-SNAPSHOT-all.jar com.google.caliper.runner.CaliperMain Tutorial
If the message changes to something like "class contains no benchmarks", then you'll know more. If you insists on using nested class, you may need to call Tutorial$Benchmark1
(unprobable, but possible; java class naming is sick).
Please try also
java -cp caliper-1.0-beta-SNAPSHOT-all.jar Tutorial.Benchmark1
to see if your class lies on the classpath (the message should change to something like "no main method").
See also this older post.