Have you seen this?
The .app is just a directory with a special structure a few special files. Once you have it created, you'll only need to update the JAR each time you build, which can be found here: MyApp.app/Contents/Resources/Java/MyJar.jar.
There will also be a few Java-specific properties in My.app/Contents/Info.plist, but you probably won't need to change these from how you initially set them up.