Stemming and partial search using MongoDB 2.4

Question

What is the correct way of doing full text search and partial searches in MongoDB? E.g. the norwegian word "sokk" (sock). When searching for "sokk" I want to match on "sokker" (sock in plural), "sokk" and "sokkepose"

A search for "sokker" should match "sokk" and "sokker".

I get the wanted result by using this ruby snippet:

def self.search(q)
    result = []

    # Full text search first
    result << Ad.text_search(q).to_a   

    # Then search for parts of the word
    result << Ad.any_of({ title: /.*#{q}.*/i }, { description: /.*#{q}.*/i} ).to_a

    result.flatten!
    result.uniq
end

Any suggestions? :)

Cheers,

Martin Stabenfeldt

Answer 1

Martin,

A few suggestions / recommendations / corrections:

1. Full Text Search in 2.4 is not production ready and should not be deployed in production without knowing the tradeoffs being made. You can find more details at - http://docs.mongodb.org/manual/tutorial/enable-text-search/

2. For Text Search to work, you need to provide appropriate language for the document while adding it (or specific fields in 2.6). This ensures the words are appropriately stemmed and stopped words are removed from indexing that field.

3. Specify language while searching for a specific field so that it is appropriately stemmed and top words removed for searching and ranking the results appropriately. You can find more details about both indexing and searching at http://docs.mongodb.org/manual/reference/command/text/ . You can also see the languages that are supported by the MongoDB FTS on that webpage.

Ideally you would not be using regular expressions while doing a full text search, but rather specify the words / strings that you are looking for along with the language.