Python BigQuery really strange timeout

Question

I am building a service to stream data into bigquery. The following code works flawlessly if i remove the part that takes 4-5 minutes to load (i am precaching some mappings)

from googleapiclient import discovery
from oauth2client import file
from oauth2client import client
from oauth2client import tools

from oauth2client.client import SignedJwtAssertionCredentials

## load email and key
credentials = SignedJwtAssertionCredentials(email, key, scope='https://www.googleapis.com/auth/bigquery')

if credentials is None or credentials.invalid:
        raw_input('invalid key')
        exit(0)

http = httplib2.Http()
http = credentials.authorize(http)

service = discovery.build('bigquery', 'v2', http=http)


## this does not hang, because it is before the long operation
service.tabledata().insertAll(...)


## some code that takes 5 minutes to execute
r = load_mappings()
## aka long operation

## this hangs
service.tabledata().insertAll(...)

If i leave the part that takes 5 minutes to execute, the Google API stops responding to the requests i do afterwards. It simply hangs in there and doesn't even return an error. I left it even 10-20 minutes to see what happens and it just sits there. If i hit ctrl+c, i get this:

^CTraceback (most recent call last):
  File "./to_bigquery.py", line 116, in <module>
    main(sys.argv)
  File "./to_bigquery.py", line 101, in main
    print service.tabledata().insertAll(projectId=p_n, datasetId="XXX", tableId="%s_XXXX" % str(shop), body=_mybody).execute()
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/googleapiclient/http.py", line 716, in execute
    body=self.body, headers=self.headers)
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 490, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1593, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1335, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1291, in _conn_request
    response = conn.getresponse()
  File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
    line = self.fp.readline()   
  File "/usr/lib/python2.7/socket.py", line 430, in readline
    data = recv(1)
  File "/usr/lib/python2.7/ssl.py", line 241, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 160, in read
    return self._sslobj.read(len)

I have managed to temporarily fix it by placing the big loading operation BEFORE the credentials authorization, but it seems like a bug to me. What am i missing?

EDIT: I have managed to get an error, while waiting:

  File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/googleapiclient/http.py", line 716, in execute
    body=self.body, headers=self.headers)
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 490, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1593, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1335, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1291, in _conn_request
    response = conn.getresponse()
  File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
    line = self.fp.readline()
  File "/usr/lib/python2.7/socket.py", line 430, in readline
    data = recv(1)
  File "/usr/lib/python2.7/ssl.py", line 241, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 160, in read
    return self._sslobj.read(len)
socket.error: [Errno 110] Connection timed out

It said timeout. This seems to happen with cold tables..

Answer 1

def refresh_bq(self):
    credentials = SignedJwtAssertionCredentials(email, key, scope='https://www.googleapis.com/auth/bigquery')

    if credentials is None or credentials.invalid:
        raw_input('invalid key')
        exit(0)

    http = httplib2.Http()
    http = credentials.authorize(http)

    service = discovery.build('bigquery', 'v2', http=http)
    self.service = service

i am running self.refresh_bq() everytime i do some inserts that do not require preprocessing, and it works flawlessly. messy hack, but i needed to make it work ASAP. There is def. a bug somewhere.