BlobInfo object from a BlobKey created using blobstore.create_gs_key

Question

I am converting code away from the deprecated files api.

I have the following code that works fine in the SDK server but fails in production. Is what I am doing even correct? If yes what could be wrong, any ideas how to troubleshoot it?

# Code earlier writes the file bs_file_name. This works fine because I can see the file
# in the Cloud Console.
bk = blobstore.create_gs_key( "/gs" + bs_file_name)
assert(bk)
if not isinstance(bk,blobstore.BlobKey):
    bk = blobstore.BlobKey(bk)
assert isinstance(bk,blobstore.BlobKey)
# next line fails here in production only
assert(blobstore.get(bk))  # <----------- blobstore.get(bk) returns None 

Answer 1

Unfortunately, as per the documentation, you can't get a BlobInfo object for GCS files.

https://developers.google.com/appengine/docs/python/blobstore/#Python_Using_the_Blobstore_API_with_Google_Cloud_Storage

Note: Once you obtain a blobKey for the GCS object, you can pass it around, serialize it, and otherwise use it interchangeably anywhere you can use a blobKey for objects stored in Blobstore. This allows for usage where an app stores some data in blobstore and some in GCS, but treats the data otherwise identically by the rest of the app. (However, BlobInfo objects are currently not available for GCS objects.)

Answer 2

I encountered this exact same issue today and it feels very much like a bug within the blobstore api when using google cloud storage.

Rather than leveraging the blobstore api I made use of the google cloud storage client library. The library can be downloaded here: https://developers.google.com/appengine/docs/python/googlecloudstorageclient/download

To access a file on GCS:

import cloudstorage as gcs

with gcs.open(GCSFileName) as f:
    blob_content = f.read()
    print blob_content

Answer 3

It sucks that GAE has different behaviours when using blobInfo in local mode or the production environment, it took me a while to find out that, but a easy solution is that:

You can use a blobReader to access the data when you have the blob_key.

def getBlob(blob_key):
  logging.info('getting blob('+blob_key+')')

  with blobstore.BlobReader(blob_key) as f:
    data_list = []
    chunk = f.read(1000)
    while chunk != "":
      data_list.append(chunk)
      chunk = f.read(1000)

    data = "".join(data_list)

  return data`

https://developers.google.com/appengine/docs/python/blobstore/blobreaderclass