Internal-use websites: Is there a compelling case against SQLite?

Question

Many web frameworks, such as Flask or Django use SQLite as their default database. SQLite is compelling because it's included in python, and administrative overhead is pretty low.

However, most high traffic public production sites wind up using a heavier database: mySQL, Oracle, or postgresql.

The questions:

Assume:

• Site traffic is moderate, and concurrent read/write access to the database will happen
• We will use SQLAlchemy with SQLite write locks (although this comment makes me a little nervous)
• The database will contain perhaps 60,000 records
• Data structures do not require advanced features found in heavier databases

Is there ever a compelling case against SQLite concurrency for websites that serve as moderate-traffic internal corporate tools? If so, what conditions will cause SQLite to have concurrency problems?

I'm looking for known specific root causes, instead of general fear / unsubstantiated finger pointing.

Answer 1

I recommend reading the official answer to your question, Appropriate Uses For SQLite. Specifically, the "Situations Where Another RDBMS May Work Better" warns that SQLite does not support concurrent writing:

SQLite supports an unlimited number of simultaneous readers, but it will only allow one writer at any instant in time. For many situations, this is not a problem. Each application does its database work quickly and moves on, and no lock lasts for more than a few dozen milliseconds. But there are some applications that require more concurrency, and those applications may need to seek a different solution.

From an appropriateness perspective, I tend to view SQLite as a very sophisticated file format that supports SQL queries. I would tend to avoid SQLite if I wanted to separate my database from my web application, as it is not optimized for this case. In short, SQLite is insufficiently scalable for use in some scenarios, so people running websites who hope to someday become popular may be better off starting with something scalable, rather than going with SQLite and later being forced to switch.

All that being said, SQLite is probably fine for most internal websites; typically internal websites don't require the same level of concurrency and scalability.

Answer 2

Putting on my IT Director hat I see a few no-gos here:

• Risk of data corruption. Perhaps more percieved than real but end of the day this is a non-transactional file type DB that does not have much if any recourse to bad writes besides asking if you've got a recent backup. Speaking of which . . .
• How do I back this thing up? In a manner I know I've got a good copy. Preferably without taking the app offline.
• How can I secure access to the DB? My general understanding is SQL lite has none outside of the filesystem access, which is a decent start but not the be-all, end-all. Especially for web apps where you might want more graduated permissions than DBA or nothing.

From a developer point of view I think it is important to know why SqlLite is the default -- it is because it is easy and it demos well. If you are "selling" a platform to new developers, being able to fire up a working web app with minimal fuss is key. And having to stand up and properly configure a database server would be a massive stumbling block to be avoided.