I figured it out this morgning:
The environment variables are not set within the virtualenv, but by another script. First you have to activate the virualenv, then you need to load the variables by activating the env script in the 'current' directory. Like this:
source /opt/python/run/venv/bin/activate
source /opt/python/current/env
The Beanstalk RDS variables are now set and ready to use by any script you execute in SSH.