get_all_keys from folder within bucket on Google Cloud Storage

Question

I need some help on getting all the objects(keys) from a FOLDER within a bucket on Google Cloud Storage. Currently, I am executing the following in Python:

GOOGLE_STORAGE = 'gs'
src_uri = boto.storage_uri(base_bucket_name + '/' + userid, GOOGLE_STORAGE)

print 'this is the src_uri: %s' % src_uri

for key in src_uri.get_all_keys():
  print 'this is the key: %s' % key

And it returns:

this is the src_uri: gs://basebucket/user2
this is the key: <Key: basebucket,user1/key1>
this is the key: <Key: basebucket,user1/key2>
this is the key: <Key: basebucket,user1/key3>
this is the key: <Key: basebucket,user1/key4>

It is returning the entire list of keys within the bucket. Though it is possible to filter out the keys belonging to other users manually, that doesn't scale and there is surely a better way to do this. Let me know if you've had any experience with this.

Answer 1

If you take a look at the documentation for get_all_keys, you need to pass the prefix and delimiter arguments.

However, I would suggest using the list function instead. Something like this should work:

bucket_uri = boto.storage_uri(base_bucket_name, GOOGLE_STORAGE)
for object_uri in bucket_uri.list_bucket(prefix='/%s' % userid, delimiter='/'):
    print object_uri