Using south to migrate database table

Question

I was not using south. Now I want to add a couple columns. Am I screwed?

(env)noah:code broinjc$ ./manage.py schemamigration reports --initial
Creating migrations directory at '/Users/broinjc/esp/code/reports/migrations'...
Creating __init__.py in '/Users/broinjc/esp/code/reports/migrations'...
 + Added model reports.Classroom
 + Added model reports.Student
 + Added model reports.SurveySet
 + Added model reports.Survey
Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate reports
(env)noah:code broinjc$ ./manage.py migrate reports
Running migrations for reports:
 - Migrating forwards to 0001_initial.
 > reports:0001_initial
FATAL ERROR - The following SQL query failed: CREATE TABLE "reports_classroom" ("id" integer NOT NULL PRIMARY KEY, "user_id" integer NOT NULL, "added" datetime NOT NULL, "updated" datetime NOT NULL, "name" varchar(30) NOT NULL, "am_or_pm" varchar(2) NOT NULL)
The error was: table "reports_classroom" already exists
 ! Error found during real run of migration! Aborting.

 ! Since you have a database that does not support running
 ! schema-altering statements in transactions, we have had 
 ! to leave it in an interim state between migrations.

! You *might* be able to recover with:   = DROP TABLE "reports_classroom"; []
   = DROP TABLE "reports_student"; []
   = DROP TABLE "reports_surveyset"; []
   = DROP TABLE "reports_survey"; []

 ! The South developers regret this has happened, and would
 ! like to gently persuade you to consider a slightly
 ! easier-to-deal-with DBMS (one that supports DDL transactions)
 ! NOTE: The error which caused the migration to fail is further up.
Error in migration: reports:0001_initial

After seeing all that, I thought, maybe I need to update my models (making them inconsistent with sqlite db) So I updated them and then ran the same command but with --auto instead of initial...

(env)noah:code broinjc$ ./manage.py schemamigration reports --auto
 ? The field 'SurveySet.top_num' does not have a default specified, yet is NOT NULL.
 ? Since you are adding this field, you MUST specify a default
 ? value to use for existing rows. Would you like to:
 ?  1. Quit now, and add a default to the field in models.py
 ?  2. Specify a one-off value to use for existing columns now

... So I went ahead with option 2, and then proceeded to migrate...

(env)noah:code broinjc$ ./manage.py migrate reports
Running migrations for reports:
 - Migrating forwards to 0002_auto__add_field_surveyset_top_num__add_field_surveyset_externalizer_ra.
 > reports:0001_initial

FATAL ERROR - The following SQL query failed: CREATE TABLE "reports_classroom" ("id" integer NOT NULL PRIMARY KEY, "user_id" integer NOT NULL, "added" datetime NOT NULL, "updated" datetime NOT NULL, "name" varchar(30) NOT NULL, "am_or_pm" varchar(2) NOT NULL)
The error was: table "reports_classroom" already exists
 ! Error found during real run of migration! Aborting.


 ! Since you have a database that does not support running
 ! schema-altering statements in transactions, we have had 
 ! to leave it in an interim state between migrations.

! You *might* be able to recover with:   = DROP TABLE "reports_classroom"; []
   = DROP TABLE "reports_student"; []
   = DROP TABLE "reports_surveyset"; []
   = DROP TABLE "reports_survey"; []

Answer 1

I'll try and explain what's going on so you better understand how to do what you want yourself.

Prior to using south you have some tables in your database which were generated from your models when you first run syncdb.

If you change your model, say you add a field "my_field", Django will fail when trying to read/write to it, since the table doesn't contain a column named "my_field". You'd normally have to dump your entire table and recreate it with syncdb. I'm sure you don't want to do that since you already have some data in you DB.

Say you want to make some changes without losing the data. First, you need to "convert" your app to south.

Basically, when you run schemamigration --initial, South will create a script (0001_initial.py) to replicate the current state of your models into a database.

If you run that script via manage.py migrate reports, it'll try to recreate all the tables you had initially, but in your case, since your DB already contains those tables, it'll scream at you saying the tables already exist:

FATAL ERROR - The following SQL query failed: CREATE TABLE "reports_classroom" ("id" integer NOT NULL PRIMARY KEY, "user_id" integer NOT NULL, "added" datetime NOT NULL, "updated" datetime NOT NULL, "name" varchar(30) NOT NULL, "am_or_pm" varchar(2) NOT NULL)
The error was: table "reports_classroom" already exists

The way to make South believe you have already applied that migration, you use the --fake option.

manage.py migrate reports 0001 --fake

Which is like saying, go to the migration state 0001_initial (you only have to write the numeric part of the name), but don't actually apply the changes.

After doing that, say you add a new field "my_field_02" to one of your models. As before, Django is referencing a field that doesn't exist in your model's table. To create it without writing the SQL yourself, you do:

manage.py schemamigration reports --auto

Which will create a new migration called something like 0002_auto__add_my_field_02.py which you then need to apply via manage.py migrate reports. You could also say manage.py migrate reports 0002 to specify the migration state you want to go to, but by default South will try to apply all the following migrations (remember you're already at state 0001).

I highly recommend you read South's documentation and backup your production data prior to doing anything.

tl;dr Read this and backup your data.