It turns out, I just needed to put it in a string (keep in mind the ORDER IS EVERYTHING, if the keys/values are out of order then it won't work). The header looks like the following
OAuth oauth_consumer_key="consumer_key", oauth_nonce="61959852", oauth_signature="signature", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1383694192", oauth_version="1.0"
My very simple code to do this is as follows:
def auth_header(creds, http_session):
auth = 'OAuth ' + 'oauth_consumer_key=\"' + creds['oauth_consumer_key']
auth = auth + '\",oauth_nonce=\"' + creds['oauth_nonce']
auth = auth + '\",oauth_signature=\"' + creds['oauth_signature']
auth = auth + '\",oauth_signature_method=\"' + creds['oauth_signature_method']
auth = auth + '\",oauth_timestamp=\"' + creds['oauth_timestamp']
auth = auth + '\",oauth_version=\"' + creds['oauth_version'] + '\"'
http_session.headers['Authorization'] = auth