Loading a Django Fixture containing Natural Keys

Question

How do you load a Django fixture so that models referenced via natural keys don't conflict with pre-existing records?

I'm trying to load such a fixture, but I'm getting IntegrityErrors from my MySQL backend, complaining about Django trying to insert duplicate records, which doesn't make any sense.

As I understand Django's natural key feature, in order to fully support dumpdata and loaddata usage, you need to define a natural_key method in the model, and a get_by_natural_key method in the model's manager.

So, for example, I have two models:

class PersonManager(models.Manager):
    def get_by_natural_key(self, name):
        return self.get(name=name)

class Person(models.Model):

    objects = PersonManager()

    name = models.CharField(max_length=255, unique=True)

    def natural_key(self):
        return (self.name,)

class BookManager(models.Manager):
    def get_by_natural_key(self, title, *person_key):
        person = Person.objects.get_by_natural_key(*person_key)
        return self.get(title=title, person=person)

class Book(models.Model):

    objects = BookManager()

    author = models.ForeignKey(Person)

    title = models.CharField(max_length=255)

    def natural_key(self):
        return (self.title,) + self.author.natural_key()
    natural_key.dependencies = ['myapp.Person']

My test database already contains a sample Person and Book record, which I used to generate the fixture:

[
    {
        "pk": null, 
        "model": "myapp.person", 
        "fields": {
            "name": "bob"
        }
    }, 
    {
        "pk": null, 
        "model": "myapp.book", 
        "fields": {
            "author": [
                "bob"
            ], 
            "title": "bob's book", 
        }
    }
]

I want to be able to take this fixture and load it into any instance of my database to recreate the records, regardless of whether or not they already exist in the database.

However, when I run python manage.py loaddata myfixture.json I get the error:

IntegrityError: (1062, "Duplicate entry '1-1' for key 'myapp_person_name_uniq'")

Why is Django attempting to re-create the Person record instead of reusing the one that's already there?

Answer 1

Turns out the solution requires a very minor patch to Django's loaddata command. Since it's unlikely the Django devs would accept such a patch, I've forked it in my package of various Django admin related enhancements.

The key code change (lines 189-201 of loaddatanaturally.py) simply involves calling get_natural_key() to find any existing pk inside the loop that iterates over the deserialized objects.

Answer 2

Actually loaddata is not supposed to work with existing data in database, it is normally used for initial load of models. Look at this question for another way of doing it: Import data into Django model with existing data?