How do I fix django.db.utils.IntegrityError: duplicate key value violates unique constraint?

Question

I'm getting the following error:

django.db.utils.IntegrityError: duplicate key value violates unique constraint "record_coordinates_lat_lon_created_by_id_key"
DETAIL:  Key (lat, lon, created_by_id)=(34.84000015258789, -111.80000305175781, 2) already exists.

Background: I've been using MySQL with Django 1.4.3 up until this point. Now I've installed Postgres 9.3 and Psycopg2 2.5.2. Validate and Syncdb worked fine. South is NOT installed.

I ran a script (which works with MySQL). The script loops through GPS files and saves lat/lon data in a coordinates table.

_coordinates, coordinates_created = Coordinates.objects.get_or_create(
    lat=round(group[u'lat'], Coordinates.max_decimal_places),
    lon=round(group[u'lon'], Coordinates.max_decimal_places),
    created_by=self._user,
    modified_by=self._user,
    date_created=datetime.now(),  # See Update 2 addition below.
)

I have a unique_together = ('lat', 'lon', ) constraint in the model definition. Some of the coordinates are identical (thus the use of get_or_create()). I find myself scratching my head because it should be 'getting' the coordinates and not attempting to 'create' new ones.

Most of the questions on this site about Postgres and Django almost inevitably mention South. Do I need South, or is something else going on here? I just wanted to do a quick and dirty test without installing migrations.

Update 1: Something else I tried was to run: SELECT setval('django_content_type_id_seq', (SELECT MAX(id) FROM django_content_type)); in Postgres on the suggestion of another post. The error persists.

Update 2: Okay I didn't realize that I needed to put all of my Coordinates fields in the defaults dict. The Coordinates model contains another field 'date_created=models.DateTimeField(auto_now_add=True)'

I found the following blog post that seems to explain that get_or_create() breaks when you use 'auto_now_add=True'. The big question is now how do I use auto_now_add and without breaking get_or_create()?

Answer 1

This answered my question.

_coordinates, coordinates_created = Coordinates.objects.get_or_create(            
    lat=Decimal(group[u'lat'])._rescale(-Coordinates.max_decimal_places, 'ROUND_HALF_EVEN'),
    lon=Decimal(group[u'lon'])._rescale(-Coordinates.max_decimal_places, 'ROUND_HALF_EVEN'),
    created_by=self._user,
    modified_by=self._user,         
)

The auto_now and auto_now_add were just fine. It turns out that all of my defaults are already defined on the Model.

The problem was that group[u'lat'] and group[u'lon'] were both cast as floats when I placed them in my dictionary. In contrast, lat and lon are both defined as DecimalFields().

When using MySQL I can compare these float values against the contents of the database just fine. However, when I use Postgres the get() portion of get_or_create() attempts to compare a Decimal value from the database with the the float values I was providing it. The types are interpreted more strongly, and the float will not be cast to a Decimal during the comparison.

In my debugger, I saw:

{Decimal}lat
{float}group[lat]

It would be nice if django could produce a clear error like TypeError: Can't compare Decimal with float.

Answer 2

You missed defaults argument in the get_or_create call. Then, if there is no record in the db with the specified lat, lon, it will create a new record with the default lat, lon, which obviously are not autogenerated and you get the IntegrityError.

_coordinates, coordinates_created = Coordinates.objects.get_or_create(
    lat=round(group[u'lat'], Coordinates.max_decimal_places),
    lon=round(group[u'lon'], Coordinates.max_decimal_places),
    created_by=self._user,
    defaults=dict(
        lat=round(group[u'lat'], Coordinates.max_decimal_places),
        lon=round(group[u'lon'], Coordinates.max_decimal_places),
        created_by=self._user,
        modified_by=self._user,
    )
)

As the unique index consist of columns lat, lon and created_by (looking at the index name from the error), you should use all of them in the filters of get_or_create.