With that configuration, all mod_wsgi is doing is adding the virtualenv site-packages directory into sys.path, nothing more.
You cannot use that as a place to set environment variables when using mod_wsgi as it is never used by mod_wsgi in any of the ways it can be configured.