Django database with router returns a count, but no records
-
05-06-2021 - |
Question
I have a django-orm database populated with data collected in spreadsheets. I am trying to write a diff function to check the current state of the database against the contents of a spreadsheet. I load the data from the spreadsheet using a router to put it into a temporary, sqlite database (the 'default' database is postgres), Here's my router:
reference_models = ['Stimulus'] # these values are fixed
load_models = ['Response', 'Participant'] # these values vary per spreadsheet
class TestRouter(object):
def db_for_read(self, model, **hints):
if model._meta.object_name in reference_models:
return 'default'
else:
assert model._meta.object_name in load_models
return 'test'
def db_for_write(self, model, **hints):
assert model._meta.object_name not in reference_models
return 'test'
def allow_relation(self, obj1, obj2, **hints):
return True
def allow_syncdb(self, db, model):
if db == "test":
return True
return None
Test from my diff script:
print("---> there are", Response.objects.all().using("default").count(), "original Response objects")
print(Response.objects.all().using("default").select_related("participant"))
print("---> there are", Response.objects.all().count(), "temporary Response objects")
print(Response.objects.all())
The output of this is:
---> there are 137709 original Response objects
[ ## LOTS OF OBJECTS ##, '...(remaining elements truncated)...']
---> there are 1680 temporary Response objects
[]
Why is this final query empty, if the .count()
method finds 1680 objects?
EDIT Here is some bonus diagnostic information (using from IPython import embed
to look at the state of the script while it is running):
In [16]: print Response.objects.all().query
SELECT "data_response"."id", "data_response"."modified", "data_response"."created",
"data_response"."stimulus_id", "data_response"."participant_id", "data_response"."full_response",
"data_response"."full_response_orthography" FROM "data_response" INNER JOIN "data_stimulus" ON
("data_response"."stimulus_id" = "data_stimulus"."id") INNER JOIN "data_participant" ON
("data_response"."participant_id" = "data_participant"."id") ORDER BY "data_stimulus"."task" ASC,
"data_stimulus"."number" ASC, "data_participant"."code" ASC
I suppose the problem is that the Stimulus object is a foreign key, and the router is getting confused where to read from. I can however pull up a Response object by id in the ipython console, and then get its Stimulus as a regular attribute (r = Response.objects.get(id=123)
then s = r.stimulus
).
another EDIT I've hacked together a functional but unsatisfying solution using:
responses = []
for response in Response.objects.raw("""select * from data_response"""):
responses.append(response)
I don't know why this works and Response.objects.all()
doesn't.
No correct solution