MySQL search FTS vs Multiple Queries

Question

Working on a project where schema is something like this:

id , key, value

The key and value columns are varchar, and the table is InnoDB.

A user can search on the basis of key value pairs ... Whats the best way to query in MySQL ? the options I can think of is:

• For each key => value form a query and perform an inner join to get id matching all criterias.

• Or in the background, populate a MyISAM table id, info with Full Text index on info and a single query using like '%key:value%key2:value2%'. The benefit of this will be later on if the website is popular and the table has a hundred thousand rows, I can easily port the code to Lucene but for now MySQL.

Answer 1

The pattern you're talking about is called relational division.

Option #1 (the self-join) is a much faster solution if you have the right indexes.

I compared the performance for a couple of solutions to relational division in my presentation SQL Query Patterns, Optimized. The self-join solution worked in 0.005 seconds even against a table with millions of rows.

Option #2 with fulltext isn't correct anyway as you've written it, because you wouldn't use LIKE with fulltext search. You'd use MATCH(info) AGAINST('...' IN BOOLEAN MODE). I'm not sure you can use patterns in key:value format anyway. MySQL FTS prefers to match words.

Answer 2

@Bill Karwin

If you're going to do this for 1 condition, it will be super fast with this EAV-like schema, but if you do it for many (esp. with mixed ANDs and ORs) it will probably fall apart. The best you can hope for is some sort of super fast index merge, and that's elusive. You're going to get a temporary table in most DBMSes if you do anything fancy. I think I remember reading you're no fan of EAV, though, and maybe I'm misunderstanding you.

As I recall, a DBMS is also free to do multiple scans and then handle this with a disposable bitmap index. But fulltext indexes keep the document lists sorted and do a low-cost merge across all criteria with a FTS planner that starts strategically with the rarer keywords. That's all they do to execute "word1 & word2" all day. They're optimized for this sort of thing.

So if you have lots of simple facts, a FTS index is one decent way to do it I think. Am I missing something? You just need to change the facts to something indexable like COLORID_3, then search for "COLORID_3 & SOMETHINGELSEID_5."

If the queries involve no merging or sorting, I suspect it will be pretty much as wash. Nothing here but us BTREEs ...