django-storages with multiple S3 Buckets

Question

I am using AWS and I will have different buckets in my application. I am also using Django-Storages. Is there a way to specify which bucket I want to upload the file to (for example, as a parameter in the Save() function or whatever?)

I saw this Django - Error importing storages.backends, but I don't understand how it should be used?!

Answer 1

S3BotoStorage takes the bucket name as a parameter. If not given it will use the AWS_STORAGE_BUCKET_NAME setting. That means if you want to make S3BotoStorage the default storage backend with DEFAULT_FILE_STORAGE then it must use the default bucket.

However you can also assign a storage on a field level:

from django.db import models
from storages.backends.s3boto import S3BotoStorage

class MyModel(models.Model):
    file_1 = models.FileField() # Uses default storage
    file_2 = models.FileField(storage=S3BotoStorage(bucket='other-bucket'))

Edit:

Comments are getting out of hand so I'll update my answer. Changing the parameters of the storage backend on an instance basis is not something that the Django storage API was designed to do. The storage backend does not have knowledge of the model instance because the storages can be used outside the context of a model such as with static files. Not completely unreasonable but it's not a usage that Django or django-storages was intended to solve. I don't expect you aren to find a drop in storage backend that will handle this for you.

The docs describe how you can manage files manually: https://docs.djangoproject.com/en/1.9/topics/files/#storage-objects At a minimum you would need store the bucket where you saved the file somewhere so that you can find it later when you query the model.

Answer 2

Another solution if you want to specify bucket on the runtime, you can do so before invoking the save() method on the model.

Following the above example:

from django.db import models
from storages.backends.s3boto import S3BotoStorage

class MyModel(models.Model):
    file_1 = models.FileField() # Uses default storage
    file_2 = models.FileField()

In views when saving the model, you can specify the storage on that field.

my_file_model = MyModel()
my_file_model.file_2.storage = S3BotoStorage(bucket="your-bucket-name")
my_file_model.save()

In this way file_2 will be saved in the bucket you specify where file_1 will use your default bucket.

Answer 3

Just mention another bucket name in settings.py with PRIVATE_BUCKET_NAME='bucket name'.

Create a custom class which override S3BotoStorage and which can be serialize into migration files.

Create a object for class s3_storage = S3MediaStorage() and give it to storage in file1 field in MyModel

from storages.backends.s3boto import S3BotoStorage
from django.conf import settings

@deconstructible
class S3MediaStorage(S3BotoStorage):
    def __init__(self, *args, **kwargs):
        kwargs['bucket'] = getattr(settings, 'PRIVATE_BUCKET_NAME')
        super(S3MediaStorage, self).__init__(*args, **kwargs)

s3_storage = S3MediaStorage()

class MyModel(models.Model):
    file = models.FileField()
    file1 = models.FileField(storage=s3_storage)

Answer 4

For S3Boto3Storage use this:

media_file = models.FileField('media file', storage=S3Boto3Storage(bucket_name='media_bucket'), upload_to='media', blank=True)