Steaming insert / insertAll - long delay?

Question

I'm streaming data into a table on BigQuery from the google api for python.

I get a 200 OK with no 'insertErrors' key in the response from bigQuery:

{u'kind': u'bigquery#tableDataInsertAllResponse'}

If I query my table there are only 2 rows, despite me having inserted several additional records a little while ago (20 minutes+).

I can't see any errors anywhere - can someone advise me on how to debug this issue?

Answer 1

I found a solution here: https://stackoverflow.com/a/19145783/1607103

I resolved the issue I was experiencing by creating a new dataset and new tables within it. Everything is now working perfectly and data is visible seconds after adding it. I am at a stage in my project where table structures are changing frequently, which seems to be the cause of the issue. I will ensure that modified tables have different names for now.

I am extremely concerned that bigquery is experiencing issues with no visibility. It doesn't fill me with confidence that there are no errors and there is no indication (that I am aware of) that something is going wrong.