iterating through S3 bucket with stream wrapper

Question

I am trying to list all the items in my Amazon S3 bucket. I have several nested directories in it.

• dir1/
• dir1/subdir1/
• dir1/subdir2/
• dir1/subdir3/
• dir2/
• dir2/subdir1/
• dir2/subdir2/
• ...

Each subdirectory contains several files. I need to get a nested array with this file structure.

I'm using the Amazon AWS SDK for PHP 2.4.2

This is my code:

$dir = 's3://bucketname';

$iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($dir));

foreach ($iterator as $file) {
    echo $file->getType() . ': ' . $file . "\n";
}

However, the result only lists files lying in the bucket, not files lying in directories/subdirectories (files with prefixes) or the directories itself.

If I iterate through ($dir.'/folder') there is no result at all.

I I pass RecursiveIteratorIterator::SELF_FIRST as the second argument to the constructor of the iterator, I get only first level directories – no subdirectories.

How can I use the AWS stream wrapper and the PHP RecursiveIterator to list all the files in all the directories in my bucket?

I hope someone can help me.

Thank you!

Answer 1

I had the same problem, and I solved this using:

use \Aws\S3\StreamWrapper;
use \Aws\S3\S3Client;

private $files = array();
private $s3path = 'YOUR_BUCKET';
private $s3key = 'YOUR_KEY';
private $s3auth = 'YOUR_AUTH_CODE';

public function recursive($path)
{
    $dirHandle = scandir($path);

    foreach($dirHandle as $file)
    {
        if(is_dir($path.$file."/") && $file != '.' && $file != '..')
        {
            $this->recursive($path.$file."/");
        }
        else
        {
           $this->files[$path.$file] = $path.$file;
        }
    }
}

public function registerS3()
{
    $client = S3Client::factory(array(
        'key'    => $this->s3key,
        'secret' => $this->s3auth
    ));

    $wp = new StreamWrapper();
    $wp->register($client);
}

public function run()
{
    $folder = 's3://'.$this->s3path.'/';

    $this->registerS3();
    $this->recursive($folder);
}

Now, if you do a DUMP in $this->files, should show all the files on the bucket.