Google App Engine and Google Big Query

Question

I am considering the Google Infrastructure and the Google App Engine for a project of mine. One thing I hoped for was that I can use the Google App Engine Datastore as Data Source for Big Query, but apparently that is not the case. Is there a recipe for doing that?

Answer 1

Google Big Query supports at the moment only csv imports. See https://developers.google.com/bigquery/docs/developers_guide#tables. You can either upload the file directly or import it from Google Cloud Storage.

What you can do is to first convert your datastore entities to a CSV file and then import that into your dataset.

Answer 2

Google Big Query has its own data repository, you need to save (append) the data onto it.
The AppEngine datastore is can be used only inside the AppEngine application.