The problem here is, because PostgreSQL does not wait untill the end of transaction to check for NOT NULL.
"NOT NULL and CHECK constraints are always checked immediately when a row is inserted or modified (not at the end of the statement)."
And my columns were set to NOT NULL.
As a workaround, I'm doing this
with transaction.atomic():
addr = Address()
addr.person_id = 0 # setting this temporarily to Zero
addr.save()
person = Person()
person.address_id = addr.id
person.save()
addr.person_id = person.id
addr.save()
This allows me to insert into the database, and update the foreign keys later, and still fail with if no foreign key matches, because I will never have a value of Zero for id value.