OK, this was a mistake on my part. The Ant file I was using to compile the code had this attribute on the javac
task:
bootclasspath="${java.home}/lib/rt.jar"
Doh. You can add the JCE jar like this:
bootclasspath="${java.home}/lib/rt.jar:${java.home}/lib/jce.jar"