Tweaking my free-text search to return the correct results - Ranking seems off at the moment

Question

I am using free-text on my website to get better search results but am getting something I don't understand.

If I use this as my free-text search (with containtstable())

isabout("*redflower*" weight (.3), "*red*" weight (.2), flower weight (.1))

I get Red Mason Bee Box with a rank or 172 and also get Wildflower Seeds - Red Campion with a rank of 172. This description is the only field being searched on. I would have thought that the Wildflower Seeds - Red Campion should rank higher because it contains BOTH terms not just one.

Could someone explain what is happening and maybe provide some examples of how to tweak it so it ranks higher for descriptions with all the words present?

EDIT: I have also tried:

isabout("*redflower*", "*red*" weight (.2), "*flower*" weight (.1))

which changed the results to Red Mason Bee Box with a rank of 21 and Wildflower Seeds - Red Campion with a rank of 21. I don't understand why that would happen because the second has both words in, not just one.

Answer 1

I am going to alter the way I search to use the NEAR operator before doing this weighted search. This should then bring back the most relevant results first.

i.e 'flower NEAR red' - This won't actually work in my example as you can't have leading wild cards but I am going to create reverse index's of all my fields and do "rewolf*" NEAR der' which will translate to Wildflower NEAR red

Answer 2

Firstly, your intent in the syntax appears to be defining a suffix term ("*flower") to adjust the ranking - I believe the nature of the SQL Server 2008 full text search engine supports prefix terms only ("wild*" would modify the ranking). But I see you have already realised this.

Secondly, the word breaker does not break 'Wildflower' into two words. There is a handy table valued function that enables you to view what the engine is doing when it applies its wordbreakers and stemmers - sys.dm_fts_parser:

select * from sys.dm_fts_parser('"Wildflower Seeds - Red Campion"', 1033, null, 0)


group_id    phrase_id   occurrence  special_term     display_term    expansion_type source_term
----------- ----------- ----------- ---------------- --------------- -------------- --------------------------------
1           0           1           Exact Match      wildflower      0              Wildflower Seeds - Red Campion
1           0           2           Exact Match      seeds           0              Wildflower Seeds - Red Campion
1           0           3           Exact Match      red             0              Wildflower Seeds - Red Campion
1           0           4           Exact Match      campion         0              Wildflower Seeds - Red Campion

An alternative solution is to manually define a Thesauras expansion from Flower to Wildflower:

<expansion>
     <sub>flower</sub>
     <sub>wildflower</sub>
</expansion>

But that fixes for this specific case only.