Rails Searching (Full text vs Semantic) for Food Products

Question

I'm trying to build a search interface for a grocery store in Rails. I have several grocery store products stored in my database, but I'm struggling to find a good way to search through them.

So far, I've used Thinking Sphinx to do full text searching of the products. This works well for some items, such as 'eggs', but has several problems for other searches, such as 'lime'.

For example, when I search for 'lime', I get the following returned items:

• Diet Sierra Mist Lemon Lime Fridge Mate Soda
• G2 Lemon Lime Low Calorie Electrolyte Beverage
• Deer Park Berry Lime Sparkling Water
• Ocean Spray Cocktail With Lime Cranberry Juice
• Vintage Seltzer Water Limn Lime
• Coke Diet With Lime Soda Contour Bottle
• Coke Diet With Lime Soda Fridge Pack
• Diet Sierra Mist Lemon Lime Fridge Mate Soda
• Refreshe Soda Diet Lemon Lime Fridge Pack
• Refreshe Soda Lemon Lime
• Refreshe Soda Lemon Lime Fridge Pack
• Sierra Mist Natural Lemon Lime Soda
• ... etc

All I really wanted was the actual fruit, which are listed as these products in my database:

• Limes Large
• Organic Limes
• Limes Key Prepacked

How can I make my search more intelligent?

It's worth noting that I do have a lot of categorical data on each product. I essentially have an entire "food tree", where each product is a root node of the tree.

For the product 'Limes Large', for example, I have the following categories:

Fruits & Vegetables > Fresh Fruit > Citrus > Limes Large

How can I better use these categories to improve my search?

In my research, I am starting to believe I need to use semantic searching instead of full text searching. I came across the Picky gem for Ruby, which looks promising, but I'm not sure if I have the right approach.

Can semantic searching help me perform a better search? Is Picky a good fit for data categorized in this way? Any other insights? Any guidance would be really appreciated. Thanks in advance.

Answer 1

As for the 'categories' you might want to think about using a tagging system also. There are a number of tagging gems out there. That way what is being searched on is roped off from any other data structures or 'trees' you have created.

In order to make it easy you could have it so that the tags that are automatically put on an item at creation are based on its location in the tree and or the items name. This makes it so could be easily changed if the need comes up, but at the same time have the ease of the tree system, while still having the tag system's power of narrow searches.

A good place to start would be bellow.

https://github.com/mbleigh/acts-as-taggable-on#readme

That way you can have something like Diet Sierra Mist Lemon Lime Fridge Mate Soda not tagged with Lemon so it wouldn't come up, but you could still tag something like Sunny D tagged with "Orange Juice." All while not making it to hard on your backend user by having default tags!

As for actually search it might just be worth it to look into using a google search system, while paying for no ads. You could use the tags as your keywords to narrow down the results. I believe this also helps with SEO while at the same time putting the search in google's hands, which means support is basically guaranteed.

Answer 2

I am the developer of Picky – so I am a bit biased ;)

Your question seems to be, if I understand correctly:

Given that I have products that contain the word "lime" in its name, and given that some of those products are not categorized as fruit, and some of those products are not categorized as fruit, but as beverages, you'd like the user to be able to filter the query so to only search for fruit.

I hope I understand correctly.

Picky's main mode is categorized search. That is, if somebody enters "lime fruit" in a query, Picky will find something like (name:lime, type:fruit), and all other combinations, eg. (company:lime, type:beverage). Or the user could already predefine through a nice interface, that he searches for fruits: "type:fruit lime", this would tell Picky to only look for results in type "fruit", and with "lime" in any category.

However, since you'd like to filter according to whether something is in a category or not, I think a faceted search is best suited for this kind of task. Picky recently got faceted search added. So you'd do something like:

picky_search.search "lime"

to display the results for lime, but also display all possible types for "lime" on the side using

picky_search.facets :type, filter: "lime"

You'd get a hash of possible types, like so: { :fruit => 3, :beverage => 150 }. Then, if the user clicked on "fruit", you'd send Picky another query, this time with the prepended filter:

picky_search.search "type:fruit lime" or picky_search.search "type:fruit name:lime" if it's already clear that the person is searching in the name for "lime".

This would only return the lime of type fruit.

This is only a quick overview, I hope it helps!