s3- boto- list files within a bucket by upload time

Question

I need to download every hour 100 newest files from s3 server.

bucketList = bucket.list(PREFIX)

The code above creates list of the files but it is not depend on the uploading time of the files, since it lists by file name?

I can do nothing with file name. It is given randomly.

Thanks.

Answer 1

How big is the list? You could sort the list on the 'last_modified' attr of the Key

orderedList = sorted(bucketList, key=lambda k: k.last_modified)
keysYouWant = orderedList[0:100]

If your list is HUGE this may not be efficient. Check out the inline docs for the list() function in boto.s3.bucket.Bucket.

Answer 2

My reading of List Objects operation documentation, suggests that objects are always listed in alphabetical order (by object key).

If you encode the creation time of each object into the object key, you may be able to achieve what you want.