Can a GAE app access Cloud Storage RESTful API without the OAuth dance?

StackOverflow https://stackoverflow.com/questions/9808854

Question

We are using the GAE Cloud Storage API to create and access files. We used gsutil to add our gae app service account to the default ACL for the bucket.

The next step is to do things like listing bucket contents. For this the RESTful API with OAuth access seems like a viable option. However, to access Cloud Storage from the taskqueue, we want to avoid the 'user consent' step in the OAuth dance. The API Console allows adding a client id for a service account for this purpose but we cant find any documentation or samples of using a service account to access the API.

So far we have looked at the Au-to-do app (requires OAuth dance) and the google-api-python-client samples. Neither of those shows access with the default service account.

Are there any examples of using an app engine service account to authorize requests to cloud storage RESTful API?

Was it helpful?

Solution

You can access Google Cloud Storage using interoperable access, with interoperable access you use the access key and a secret key to access your buckets.
The process is described in great details, and you can see sample code that implements the process in the boto package (look for HmacAuthV1Handler in auth.py)

OTHER TIPS

I'm in the process of publishing a small sample which illustrates precisely what you're asking about: how to use App Engine service account credentials to access Google Cloud Storage (specifically, to list a formatted Google Cloud Storage bucket). When available, this will be part of the https://code.google.com/p/google-api-python-client/ repo but, in the meantime, here's the code: 

import httplib2
import logging
import os
import pickle
import re

from google.appengine.api import memcache
from google.appengine.ext import webapp
from google.appengine.ext.webapp import template
from google.appengine.ext.webapp.util import run_wsgi_app
from oauth2client.appengine import AppAssertionCredentials

# Constants for the XSL stylesheet and the Google Cloud Storage URI.
XSL = '\n<?xml-stylesheet href="/listing.xsl" type="text/xsl"?>\n';
URI = 'http://commondatastorage.googleapis.com'

# Obtain service account credentials and authorize HTTP connection.
credentials = AppAssertionCredentials(
    scope='https://www.googleapis.com/auth/devstorage.read_write')
http = credentials.authorize(httplib2.Http(memcache))

class MainHandler(webapp.RequestHandler):

  def get(self):
    try:
      # Derive desired bucket name from path after domain name.
      bucket = self.request.path
      if bucket[-1] == '/':
        # Trim final slash, if necessary.
        bucket = bucket[:-1]
      # Send HTTP request to Google Cloud Storage to obtain bucket listing.
      resp, content = http.request(URI + bucket, "GET")
      if resp.status != 200:
        # If error getting bucket listing, raise exception.
        err = 'Error: ' + str(resp.status) + ', bucket: ' + bucket + \
              ', response: ' + str(content)
        raise Exception(err)
      # Edit returned bucket listing XML to insert style sheet for nice 
      # formatting and send results to client.
      content = re.sub('(<ListBucketResult)', XSL + '\\1', content)
      self.response.headers['Content-Type'] = 'text/xml'
      self.response.out.write(content)
    except Exception as e:
      self.response.set_status(404)
      self.response.out.write(str(e))

def main():
  application = webapp.WSGIApplication(
      [
       ('.*', MainHandler),
      ],
      debug=True)
  run_wsgi_app(application)

if __name__ == '__main__':
  main()

This code depends on an XSLT style sheet. You can comment out references to XSL to eliminate that dependency or include the following file in your project (with name listing.xsl):

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" 
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:ama="http://doc.s3.amazonaws.com/2006-03-01">

<xsl:template match="/">
  <html>
  <body>
  <h2><a href="http://developer.google.com/storage">Google Cloud Storage</a> Con
tent Listing for Bucket 
      <xsl:value-of select="ama:ListBucketResult/ama:Name"/></h2>
    <table border="1" cellpadding="5">
      <tr bgcolor="#9acd32">
        <th>Object Name</th>
        <th>Modification Time</th>
        <th>ETag</th>
        <th>Size</th>
        <th>Storage Class</th>
      </tr>
      <xsl:for-each select="ama:ListBucketResult/ama:Contents">
      <tr>
        <td><xsl:value-of select="ama:Key"/></td>
        <td><xsl:value-of select="ama:LastModified"/></td>
        <td><xsl:value-of select="ama:ETag"/></td>
        <td><xsl:value-of select="ama:Size"/></td>
        <td><xsl:value-of select="ama:StorageClass"/></td>
      </tr>
      </xsl:for-each>
    </table>
  </body>
  </html>
</xsl:template>
</xsl:stylesheet>

I assume you have seen this? http://code.google.com/appengine/docs/python/googlestorage/

I have not used this API, but have used other APIs with OAuth on App Engine by storing an authenticated OAuth token in the datastore. A similar solution may work for you also

The easiest way to use the REST API without doing the OAuth dance would be to use GAE App Identity to get an Access Token, and then use the access token to access Google Cloud Storage. In either case, you'll need to add the email form of the App Identity to your Google Cloud Storage project, as described in the prerequisites section of the Files API for Google Cloud Storage documentation (you'll need to do this even though you aren't using the Files API, since this sets up your project so that the App Engine application can access it).

Google now offers a Signed URLs (Query String Authentication) mode to access Cloud Storage buckets. This allows service accounts to skip the OAuth dance and access bucket info with a signed URL. Check it out.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top