Authenticating Service Accounts on Google Compute Engine with BigQuery, via R Studio Server

Question

I'm looking to call BigQuery from R Studio, installed on a Google Compute Engine.

I have the bq python tool installed on the instance, and I was hoping to use its service accounts and system() to get R to call bq command line tool and so get the data.

However, I run into authentication problems, where it asks for a browser key. I'm pretty sure there is no need to get the key due to the service account, but I don't know how to construct the authetication from with R (it runs on RStudio, so will have multiple users)

I can get an authetication token like this:

library(RCurl)
library(RJSONIO)
metadata <-  getURL('http://metadata/computeMetadata/v1beta1/instance/service-accounts/default/token')
tokendata <- fromJSON(metadata)

tokendata$$access_token

But how do I then use this to generate a .bigqueryrc token? Its the lack of this that triggers the authetication attempt.

This works ok:

system('/usr/local/bin/bq')

showing me bq is installed ok.

But when I try something like:

   system('/usr/local/bin/bq ls')

I get this:

Welcome to BigQuery! This script will walk you through the process of initializing your .bigqueryrc configuration file.

First, we need to set up your credentials if they do not already exist.


******************************************************************
** No OAuth2 credentials found, beginning authorization process **
******************************************************************

Go to the following link in your browser:

https://accounts.google.com/o/oauth2/auth?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fbigquery&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&client_id=XXXXXXXX.apps.googleusercontent.com&access_type=offline

Enter verification code: You have encountered a bug in the BigQuery CLI. Google engineers monitor and answer questions on Stack Overflow, with the tag google-bigquery:     http://stackoverflow.com/questions/ask?tags=google-bigquery

etc.

Edit:

I have managed to get bq functioning from RStudio system() commands, by skipping the authetication by logging in to the terminal as the user using RStudio, autheticating there by signing in via the browser, then logging back to RStudio and calling system("bq ls") etc..so this is enough to get me going :)

However, I would still prefer it if BQ can be autheticated within RStudio itself, as many users may log in and I'll need to autheticate via terminal for all of them. And from the service account documentation, and the fact I can get an authetication token, hints at this being easier.

Answer 1

For the time being, you need to run 'bq init' from the command line to set up your credentials prior to invoking bq from a script in GCE. However, the next release of bq will include support for GCE service accounts via a new --use_gce_service_account flag.