List all objects in Amazon S3 using perl

Question

I am using Perl script to delete objects/files from Amazon S3 which are older than certain period. But the problem is Perl module Amazon::S3 shows only 1000 objects. There are more than 18000 objects now in my account in amazon and i never get past the first 1000. Is there any way to list all the 18000 objects?

The code used to list 1000 objects is this:

 my $bucket = $s3->bucket("<bucket_name>");

 $response = $bucket->list or die $s3->err . ": " . $s3->errstr;
 print $response->{bucket}."\n";

 # This for loop lists only 1000 objects:

 for my $key (@{ $response->{keys} }) 
 {
       #print "\t".$key->{key}."\n";
       #delete key from bucket
       $bucket->delete_key($key->{key});
 }

Answer 1

You can try,

my $response = $bucket->list_all;

From perldoc Amazon::S3::Bucket

list_all

List all keys in this bucket without having to worry about 'marker'. This may make multiple requests to S3 under the hood. See "list_bucket_all" in Amazon::S3 for documentation of this method.

Answer 2

Amazon S3 by defaults responds with first 1000 files. take the last file name and pass it as a marker to show the next 1000 files.

$s3->list_bucket({bucket=>"bucketname", marker=>"directory/last_file"});