At what point should you switch to release build?

Question

One of the practices set out in Jez Humble's Continuous Delivery is that you should build one package and then release it to each environment you deploy to, so that the deployment and the artifacts have themselves been tested multiple times before going to production.

I fully support this idea.

On the other hand, debug-mode builds that give you stack traces with line numbers are incredibly useful in test environments, as is the ability to remote-debug. But, you want to send a release build to production.

So, for people following the first principle, at what point do you switch from debug to release builds?

Is it before the first deployment to a test environment, figuring the cost of losing debug mode is worth paying to ensure you are testing the actual release candidate early? Or do you build again at some point in the promotion process, figuring that you'll trust the build process over the software? Or do you just screw it all and deploy debug versions to production?

Note: I know this doesn't really apply to interpreted languages because you can usually flick the switch in configuration rather than doing it at build time.