unixODBC issue on a Django site on Linux that goes away on refresh

Question

I have a question about Django, unixODBC, FreeTDS, Apache2, mod_wsgi, that is somewhat similar to this question asked before on SO.

I have a Django website built to run on the latest Django, that is, 1.2.3. It uses managed models for the most part, in that, save for the session information, nothing is written to the DB by Django, only read from it.

The DB in question is hosted on a Windows machine on MSSQL 2005.

The Django project is hosted on a Linux machine. It is served over mod_wsgi on Apache2. The DB connection is through the infamous FreeTDS and unixODBC duo. Latest versions of both FreeTDS and unixODBC are running. On the Python end, pyodbc, and django-pyodbc are used for the DB stuff.

The project is deployed on two Linux live servers, with identical setups, that are behind a load balancer. There is one DB server that both connect to.

On the live, production servers, there are some intermittent issues. On the pages that query the DB to get records to display, sometimes, and I mean sometimes, an exception is thrown, with the error:

('IM001', '[IM001] [unixODBC][Driver Manager]Driver does not support this function (0) (SQLColAttribute)')

One of the strange bits about this conundrum is that whenever you get this error, simply hitting refresh on the web browser clears it and renders the page with the records fetched from the DB.

The generated SQL, from the debug output when the error occurs, is very simple:

SELECT COUNT(*) FROM [TABLE] WHERE ([TABLE].[category] = 5 AND [TABLE].[newRelease] = 1 )

On the Django development server, this doesn't show up, of course. On the development machine, with mod_wsgi running in daemon mode, I came across this error a number of times. I tested by deploying on mod_python, where I didn't see any errors. I then tested with mod_wsgi in embedded mode, where, as well, I didn't see any errors. I think changed the setup on the live server to use mod_wsgi in embedded mode, but the errors still crop up sometimes.

I am really stuck with this. I am not sure what to try to get closer to figuring out what's really causing the problem. I am not sure whether I have provided all helpful information. If I haven't, please do point it out, and I will update the question.

I don't believe there is anything in the code that is wrong and causing this. It wouldn't work most or all of the time, if it were the case. I've tried clearing .pyc files, restarting Apache, etc, on the live servers, but to no avail.

Any, any help will be very much appreciated.

Thanks.

UPDATE: I tacked the "never_cache" decorator to most of the view functions, hoping that perhaps whatever little model-level caching that Django does could be turned off. But that didn't really do anything on the live server. I am really out of ideas now.

UPDATE #2: I threw in some logging inside sql_sever/pyodbc/base.py (django-pyodbc) around the code where the exception is generated. And I've got more SQL queries for which, apparently, the elusive error was generated:

sql = SELECT * FROM (SELECT [TABLE].[id], [TABLE].[productID], [TABLE].[title], [TABLE].[price], [TABLE].[rrp], [TABLE].[saving], [TABLE].[hmvPoints], [TABLE].[availability], [TABLE].[shipping], [TABLE].[rating], [TABLE].[thumbnail], [TABLE].[details], [TABLE].[images], [TABLE].[certImage], [TABLE].[trackListing], [TABLE].[category], [TABLE].[subCategory], [TABLE].[genreId], [TABLE].[bestSeller], [TABLE].[preOrder], [TABLE].[newRelease], (ROW_NUMBER() OVER (ORDER BY [TABLE].[id] ASC)) AS [rn] FROM [TABLE] WHERE [TABLE].[productID] = ? ) AS X WHERE X.rn BETWEEN 1 AND 21

params = (799742,) 

exception = ('IM001', '[IM001] [unixODBC][Driver Manager]Driver does not support this function (0) (SQLColAttribute)')

sql = SELECT * FROM (SELECT (1) AS [a], (ROW_NUMBER() OVER (ORDER BY RAND() )) AS [rn] FROM [django_session] WHERE [django_session].[session_key] = ? ) AS X WHERE X.rn BETWEEN 1 AND 1

params = ('e4b669b40d10c336d62c8435198bf1db',)

exception = ('IM001', '[IM001] [unixODBC][Driver Manager]Driver does not support this function (0) (SQLColAttribute)')

Answer 1

I am going to make an attempt to answer the question myself. I am not too happy with this answer as it does not go into the details explaining the problem, which is in part due to the fact that I have not been able to figure out exactly why the problem occurs.

Testing by whatever means I could come up with, I was able to reproduce the problem in my development environment, and reproduce it whenever I wanted to. For some reason, which could be either due to a bug in FreeTDS or some sort of a limitation that I've not yet come across in the documentation anywhere, the FreeTDS driver starts throwing up the errors mentioned in the question above when concurrent requests are made to the application which in turn hits the database via FreeTDS. Even with as few as two users browsing the application simultaneously, the FreeTDS driver starts throwing the errors around everywhere.

What is interesting, perhaps, though is that, in my testing, I did not find it to have any issues when concurrent INSERT SQL calls were being made. The problem in question manifests itself loudly on mainly SELECT calls, to the extent that my testing goes. This discovery may only be the tip of the iceberg, for all I know. But it is common knowledge that FreeTDS is not a production-ready driver.

I have been evaluating the commercial MSSQL driver from Easysoft, SQL Server ODBC driver. Having put it through the same set of tests I did FreeTDS, I am happy with what I have here. It works really well, and of course, it is worth mentioning here that it supports a slew of features, including the important proper Unicode support, which FreeTDS does not.