SQLite: Performance of Rows VS Columns

Question

I have a database table where each row (movie) has a couple of numeric tags (movie categories). Currently I put all these tags in the same column as a string, and search for them using %LIKE%, which requires a slow full table scan when I need to find all movies in a certain category.

I want to speed up searching for these tags, but the only solution I can think of is creating a second table with two integer columns. The first one contains a single category, and the second contains the rowid of the movie.

However, this will require much more inserts in the database. A row has 10 tags on average, so instead of inserting a single row, I have to insert 11 rows. Since my application does much more inserting than actually querying, the insert-performance is crucial.

Is there another way to solve this, without sacrificing insert-performance? Or is there no big difference between inserting 1 row with 10 columns VS 10 rows with 2 columns?

Answer 1

You'll have slightly slower insert performance because the indexes need to be updated (at the very least it'll have an index on ROWID, and you need an index on category ID to get a significant speedup). The data size itself is trivial.

However, I'd expect it to be completely dwarfed by transaction overheads (all the calls to fsync(), for one). SQLite is terrible for concurrent write-heavy loads.

If you do more inserting than querying, you may want to rethink your data structure.