Collecting and Aggregating App Data on Google Gloud Platform

Question

I'm trying to collect log/app data from my android app using the Google cloud platform. Ultimately, I want to store my data in Big Query, by ingesting a csv or json file from Google Cloud Storage on a nightly basis. My question is, how can I generate the csv that contains information from multiple users of the app (essentially just combining CSV files)? Are there any libraries/sample projects that can help with this?

Furthermore, I want to use this type of flow that does not require user authentication, essentially by having a server take in the reported data from app instances and send that to cloud storage.

Answer 1

No need to convert to CSV or JSON just to import from GCS to BQ. You can write directly from your App Engine app to BQ - here is an article demonstrating how to do that with a MapReduce.