Importing CouchBase dump to BigQuery

Question

After creating a CouchBase dump and copying it to a Google Storage bucket I tried importing the data to BigQuery.

I failed miserably, apparently the design.json or the files that appear in the same directory as data-0000.cbb are not working when I'm trying the json import.

Do I need to convert the dump somehow? Is there any other way to migrate the data from CB to BQ?

Thank you all in advance.

Answer 1

I managed to export the couchbase data to CSV, after converting to CSV I discovered the files were corrupted and there were some entries I had to give up on.

When I first tried there was no "Ignore Errors" option in BigQuery so I couldn't import the files at all, after several weeks I tried again and found the "Ignore Errors" option, I immediately imported my files, first each one of them to a single DB, later on combined them together, it looks great now but it was not tested for compliance yet, I lost several records (a little less than 20) out of 80 gigs of data.