Django/South: save() not working as expected when adding a foreign key to a new table

Question

I've just started to study the South framework, waiting for the 1.7 release of Django to be released and production-ready.

I'm in this starting situation:

class TableA(models.Model):
    CustomEntityA_ctype = models.ForeignKey(ContentType, related_name="tableB_related_name")    
    CustomEntityA_oid   = models.PositiveIntegerField()
    CustomEntityA       = generic.GenericForeignKey('CustomEntityA_ctype',
                                                    'CustomEntityA_oid')

I'd like these scheme to be migrated into something like this

class TableB(models.Model):
    CustomEntityB_ctype = models.ForeignKey(ContentType, related_name="tableB_related_name")    
    CustomEntityB_oid   = models.PositiveIntegerField()
    CustomEntityB       = generic.GenericForeignKey('CustomEntityB_ctype',
                                                    'CustomEntityB_oid')

class TableA(models.Model):
    tableB_entity  = models.ForeignKey(TableB, 
                                       related_name='tableA_related_name',
                                       null=False)        

In order to get this, I've setup a proper initial migration, then my intermediate scheme is something like this

class TableB(models.Model):
    CustomEntityB_ctype = models.ForeignKey(ContentType, related_name="tableB_related_name")    
    CustomEntityB_oid   = models.PositiveIntegerField()
    CustomEntityB       = generic.GenericForeignKey('CustomEntityB_ctype',
                                                    'CustomEntityB_oid')

class TableA(models.Model):
    CustomEntityA_ctype = models.ForeignKey(ContentType, related_name="tableB_related_name")    
    CustomEntityA_oid   = models.PositiveIntegerField()
    CustomEntityA       = generic.GenericForeignKey('CustomEntityA_ctype',
                                                    'CustomEntityA_oid')

    tableB_entity  = models.ForeignKey(TableB, 
                                       related_name='tableA_related_name',
                                       null=True)        

As for South tutorial, I'm trying to split migration in three parts:

1. A first --auto migration towards this intermediate model
2. A datamigration generating the migration python script
3. A final --auto migration towards the final model

This is the content of my forwards function

def forwards(self, orm):
    "Write your forwards methods here."
    # Note: Don't use "from appname.models import ModelName". 
    # Use orm.ModelName to refer to models in this application,
    # and orm['appname.ModelName'] for models in other applications.
    for tableA_entity in orm.TableA.objects.all():
        ctype = tableA_entity.CustomEntityA_ctype
        oid   = tableA_entity.CustomEntityA_oid

        tableB_entity = orm.TableB.objects.create(CustomEntityB_ctype=ctype,
                                                  CustomEntityB_oid=oid,
                                                  )

        print "created a tableB_entity", tableB_entity
        tableA_entity.tableB_entity = tableB_entity
        tableA_entity.save()

        tableA_entity_FROMDB = orm.TableA.objects.all()[0]

        print "tableA_entity_FROMDB.tableB_entity: ", tableA_entity_FROMDB.tableB_entity

When I invoke the migration step, I get a correctly created and printed tableB_entity, but when I print the result of the query with the last two rows, I get an empty result. The overall result is that the save() function seems not to be working at all. If I enter the manage.py shell and query the models I get the expected result in TableB but an empty foreign key for the corresponding entity in TableA.

Is there anyone who might explain this to me?

Thanks a lot in advance!

Answer 1

The only way I found to let it work so far was to split the scheme and data migration in two different sets of migrations.

1. In the first one I schema/data migrated towards TableB having those three more fields.
2. In the second one, I added the foreign key to TableA and assigned the TableB entity created at the previous step to the corresponding TableA entity.

Obviously, this is not optimal: if in the first step an entity target of a many-to-one relation should be created, during the second migration it would not be straightforward which entity of TableA should point to the newly created item. A way to mitigate this might be creating a temporary reverse foreign key from TableB to TableA useful to find the original TableA entity which to refer to and then remove it in a further migration.

Before accepting this, I'll wait some more time in order to let someone else come up with the "correct" answer :)