Yes you can tweak some things. You can create a custom build for example by using the following link: http://dojotoolkit.org/documentation/tutorials/1.9/build/. I'm not sure if that will help that much (didn't follow it completely), but there's also a web tool that allows you to create custom builds and compress them into a single JS file (according to a colleague of mine). Try the Dojo web builder, I think it would probably help you a lot.
I'm not sure if you did already, but did you define async: true
in your dojoConfig? I'm not sure if it will help, but you can always try it.