Integrity error - get_or_create says cannot be NULL on importing data

Question

I asked a question to try and fix an issue I was having where I didn't understand how to use ForeignKeys properly. That was very helpfully fixed, although I had a subsequent problem with django.db.utils.IntegrityError: core_team.blahblah_id may not be NULL and I decided to roll back, do something slightly simpler - in order to avoid a double lookup (Match is linked to both Team and League), I would write a management command to import the teams.

On my (clearly flawed) understanding from the previous question, I've done it right - it uses get_or_create to check for the league instance and then assigns the team based on that. I've also doublechecked that the DB is up to date (I'm running south and did the forward migration the last time I changed the scheme, nothing's changed since then. Last change was to make the names in both models the primary key (as there's only one team of each name, only one league of each name.)

Most recently, I've added code to provide the default to the team get_or_create section, but am receiving the same error. I understand the cause (I think) of the error - that the ForeignKey 'league' in Team already exists in the database, and can't be null inserting another team (from https://docs.djangoproject.com/en/1.5/ref/models/querysets/#get-or-create), just not how to fix it.

Management command:

from django.core.management.base import BaseCommand, CommandError
import csv
import csvImporter
#from core.models import Match
from time import strptime
from datetime import datetime

master_data = open ('/Users/chris/Desktop/AllDataTruncated.csv', 'r') 
data = list(tuple(rec) for rec in csv.reader(master_data, delimiter=','))

from core.models import League, Team

team_list = []
for row in data:

    if row[2] == "HomeTeam":
        print "Continuing"
        continue
    elif row[2] == "":
        print "Continuing"
        continue
    else:
        league, _ = League.objects.get_or_create(name=row[0])
        print league
        team, _ = Team.objects.get_or_create(team_name=row[2], defaults={'league':league})
        current_team = Team(league = league, team_name=team)

    print current_team 

And relevant bits of models.py:

class League (models.Model):
    name = models.CharField(max_length=2, primary_key=True)
    last_modified = models.CharField(max_length=50)
    def __unicode__(self):
        return unicode(self.name)

class Team(models.Model):
    team_name = models.CharField(max_length=50, primary_key=True)
    league = models.ForeignKey(League)
    team_colour = models.CharField(max_length=6, null=True, blank=True)
    def __unicode__(self):
        return unicode (self.team_name)

The full traceback is:

$ python manage.py importteams
Continuing
E0
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 453, in execute_from_command_line
    utility.execute()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 272, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 77, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/Users/chris/Dropbox/Django/gmblnew/core/management/commands/importteams.py", line 26, in <module>
    team2, _ = Team.objects.get_or_create(team_name=row[3])
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/manager.py", line 146, in get_or_create
    return self.get_query_set().get_or_create(**kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 487, in get_or_create
    six.reraise(*exc_info)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 477, in get_or_create
    obj.save(force_insert=True, using=self.db)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/base.py", line 546, in save
    force_update=force_update, update_fields=update_fields)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/base.py", line 650, in save_base
    result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/manager.py", line 215, in _insert
    return insert_query(self.model, objs, fields, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 1661, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 937, in execute_sql
    cursor.execute(sql, params)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/util.py", line 41, in execute
    return self.cursor.execute(sql, params)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 364, in execute
    six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 362, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.IntegrityError: core_team.league_id may not be NULL

Now - I think that it's odd that it's saying league_id here, as this shouldn't be relevant anymore? When I did the migration, the question came up:

 ? The field 'League.id' does not have a default specified, yet is NOT NULL.
 ? Since you are removing this field, you MUST specify a default
 ? value to use for existing rows. Would you like to:
 ?  1. Quit now.
 ?  2. Specify a one-off value to use for existing columns now
 ?  3. Disable the backwards migration by raising an exception; you can edit the migration to fix it later
 ? Please select a choice: 3

Is this what's perpetuating this issue?

Edit: Seems not. Dropped the DB and moved the South migrations folder, and it's still doing it. The source CSV is also fine (no blank lines or empty strings/segments), and the code above has a section to skip those segments anyway; it's not getting that far.

Answer 1

Ugh. The answer to this, for any other newbies who are coming to it later, is actually ludicrously simple. What I'm doing here is creating an entry in the table 'Team', which has a ForeignKey going back to 'League'.

The 'trick' (it's not a trick, just really badly explained in the documentation, imho) is that you need to explicitly pass the league back when you do the get_or_create for the Team object. It's not just about matching the team name

I thought I'd done this, but I hadn't, it appears. This code works (and quite effectively ensures there are no duplicates):

for row in data:

    if row[2] == "HomeTeam":
        print "Continuing"
        continue
    elif row[2] == "":
        print "Continuing"
        continue
    else:
        league, _ = League.objects.get_or_create(name=row[0])
        print league
        team, _ = Team.objects.get_or_create(team_name=row[2], league=league)
        current_team = Team(league = league, team_name=team)
    print current_team