BigQuery Throwing Import Error, No Information Provided

Question

I am trying to import a CSV file into my BigQuery Table. This import has worked in the past, but now I am getting the following error message:

{"message":"Too many errors encountered. Limit is: 0.","reason":"invalid"}

All other fields are empty when I run the debugger.

This is... not helpful. I am unaware of any issues with the data itself, as the export/import data has not changed. Curiously, when trying to use a previous Job Template and run through the web console, the web console itself hangs and the dialog never goes away once I hit the blue "Submit" button.

Job Id: job_e0faf560d3df424ea74519e1b24a23f7

I am generating a CSV and exporting it to Google Cloud Storage. I am using AppEngine and have switched to the new Google Cloud Storage Client Library. I had uploaded the file using the GcsFileOptions.getDefaultInstance() as well as constructing my own GSFileOptions setting the content type to CSV.

After failure, I downloaded the file from Google Cloud Storage, change the encoding (tried ASCII and UTF8) and still have gotten the same result.

I am using AppEngine 1.8.1.1 and the BigQuery Library (google-api-services-bigquery-v2-rev89-1.15.0-rc). This was working as expected previously, so I'm not sure what has happened. Any suggestions are welcome. Thank you!

Answer 1

There are two error fields on the bigquery job. The first is the error result, which tells you whether (and why) the job failed. The error result in your case that the job failed due to encountering too many input errors during the import.

The second field is the error stream, which tells you about errors encountered during the job. If you had set the maxBadRecords field, for example, you could have errors in the error stream, but the actual job might succeed.

I looked up your job in the BigQuery logs, and was able to find that the error stream indicates an error on line 6253: "Too few columns: expected 80 column(s) but got 1 column(s). For additional help: http://goo.gl/RWuPQ"

Can you verify that line 6253 is correct?

-- Jordan Tigani / BigQuery Engineer

Answer 2

Today there is some general problem with app engine: "We are still investigating the issue with Google App Engine, primarily (but not restricted to) Datastore latency. We will provide another status update in the next two hours." https://groups.google.com/forum/#!topic/google-appengine-downtime-notify/1pJZnl4EMKk