Elastic Search to search for words that starts with phrase

Question

I'm trying to create a search function for my website using Elastic Search and NEST. You can see my code below and I get results if I search for complete (and almost comlete) words. Ie, if I search for "Buttermilk" or "Buttermil" I get a hit on my document containing the word "Buttermilk".

However, what I try to accomplish is if I search for "Butter", I should have a result with all three documents which have words that starts with "Butter". I thought this was solved by using FuzzyLikeThis?

Can anyone see what I'm doing wrong and point me in the right direction?

I created a console-app and the complete code you can see here:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Nest;
using Newtonsoft.Json;

namespace ElasticSearchTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var indexSettings = new IndexSettings();
            indexSettings.Analysis.Analyzers["text-en"] = new SnowballAnalyzer { Language = "English" };

            ElasticClient.CreateIndex("elastictesting", indexSettings);

            var testItem1 = new TestItem {
                Id = 1,
                Name = "Buttermilk"
            };
            ElasticClient.Index(testItem1, "elastictesting", "TestItem", testItem1.Id);

            var testItem2 = new TestItem {
                Id = 2,
                Name = "Buttercream"
            };
            ElasticClient.Index(testItem2, "elastictesting", "TestItem", testItem2.Id);

            var testItem3 = new TestItem {
                Id = 3,
                Name = "Butternut"
            };
            ElasticClient.Index(testItem3, "elastictesting", "TestItem", testItem3.Id);

            Console.WriteLine("Write search phrase:");
            var searchPhrase = Console.ReadLine();
            var searchResults = Search(searchPhrase);

            Console.WriteLine("Number of search results: " + searchResults.Count());
            foreach (var item in searchResults) {
                Console.WriteLine(item.Name);
            }

            Console.WriteLine("Press any key to exit");
            Console.ReadKey();
        }

        private static List<TestItem> Search(string searchPhrase)
        {
            var query = BuildQuery(searchPhrase);

            var result = ElasticClient
                .Search(query)
                .Documents
                .Select(d => d)
                .Distinct()
                .ToList();

            return result;
        }

        public static ElasticClient ElasticClient
        {
            get
            {
                var localhost = new Uri("http://localhost:9200");
                var setting = new ConnectionSettings(localhost);
                setting.SetDefaultIndex("elastictesting");
                return new ElasticClient(setting);
            }
        }

        private static SearchDescriptor<TestItem> BuildQuery(string searchPhrase)
        {
            var querifiedKeywords = string.Join(" AND ", searchPhrase.Split(' '));

            var filters = new BaseFilter[1];

            filters[0] = Filter<TestItem>.Bool(b => b.Should(m => m.Query(q =>
                q.FuzzyLikeThis(flt =>
                    flt.OnFields(new[] {
                        "name"
                    }).LikeText(querifiedKeywords)
                    .PrefixLength(2)
                    .MaxQueryTerms(1)
                    .Boost(2))
                )));

            var searchDescriptor = new SearchDescriptor<TestItem>()
                .Filter(f => f.Bool(b => b.Must(filters)))
                .Index("elastictesting")
                .Type("TestItem")
                .Size(500);

            var jsons = JsonConvert.SerializeObject(searchDescriptor, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore });
            return searchDescriptor;
        }
    }

    class TestItem {
        public int Id { get; set; }
        [ElasticProperty(Analyzer = "text-en", Index = FieldIndexOption.analyzed)]
        public string Name { get; set; }
    }
}

---

Edited 2014-04-01 11:18

Well, I ended up using MultiMatch and QueryString, so this it how my code looks now. Hope it mey help anyone in the furure. Also, I added a Description property to my TestItem to illustrate multimatch.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Nest;
using Newtonsoft.Json;

namespace ElasticSearchTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var indexSettings = new IndexSettings();

            ElasticClient.CreateIndex("elastictesting", indexSettings);

            var testItem1 = new TestItem {
                Id = 1,
                Name = "Buttermilk",
                Description = "butter with milk"
            };
            ElasticClient.Index(testItem1, "elastictesting", "TestItem", testItem1.Id);

            var testItem2 = new TestItem {
                Id = 2,
                Name = "Buttercream",
                Description = "Butter with cream"
            };
            ElasticClient.Index(testItem2, "elastictesting", "TestItem", testItem2.Id);

            var testItem3 = new TestItem {
                Id = 3,
                Name = "Butternut",
                Description = "Butter with nut"
            };
            ElasticClient.Index(testItem3, "elastictesting", "TestItem", testItem3.Id);

            Console.WriteLine("Write search phrase:");
            var searchPhrase = Console.ReadLine();
            var searchResults = Search(searchPhrase);

            Console.WriteLine("Number of search results: " + searchResults.Count());
            foreach (var item in searchResults) {
                Console.WriteLine(item.Name);
                Console.WriteLine(item.Description);
            }

            Console.WriteLine("Press any key to exit");
            Console.ReadKey();
        }

        private static List<TestItem> Search(string searchPhrase)
        {
            var query = BuildQuery(searchPhrase);

            var result = ElasticClient
                .Search(query)
                .Documents
                .Select(d => d)
                .Distinct()
                .ToList();

            return result;
        }

        public static ElasticClient ElasticClient
        {
            get
            {
                var localhost = new Uri("http://localhost:9200");
                var setting = new ConnectionSettings(localhost);
                setting.SetDefaultIndex("elastictesting");
                return new ElasticClient(setting);
            }
        }

        private static SearchDescriptor<TestItem> BuildQuery(string searchPhrase)
        {
            var searchDescriptor = new SearchDescriptor<TestItem>()
                .Query(q => q
                    .MultiMatch(m =>
                    m.OnFields(new[] {
                        "name",
                        "description"
                    }).QueryString(searchPhrase).Type(TextQueryType.PHRASE_PREFIX)
                    )
                )
                .Index("elastictesting")
                .Type("TestItem")
                .Size(500);

            var jsons = JsonConvert.SerializeObject(searchDescriptor, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore });

            return searchDescriptor;
        }
    }

    class TestItem {
        public int Id { get; set; }
        public string Name { get; set; }
        public string Description { get; set; }
    }
}

Answer 1

Instead of using FuzzyLikequery.. use prefix query its more fast and accurate..! for more information refer

curl -XPOST "http://localhost:9200/try/indextype/_search" -d'
{
"query": {
    "prefix": {
       "field": {
          "value": "Butter"
       }
    }
}
}'

create above query in NEST and try again..!

Answer 2

This has nothing to do with FuzzyLikeThis.

You can use prefixquery as suggested by @BlackPOP out of the box. You could also opt for using EdgeNGrams, this will tokenize your input on index-time. The result faster performance as compared to prefixquery, offset against increased index size.

One thing to keep in mind is that prefixquery only works on non-analyzed fields, so if you want to do any anaylzing at indexing-time, you're probably better off using EdgeNGrams.

Please read up on anaylzers etc, if you don't know what they are. Some refs:

• http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-analyzers.html
• http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-edgengram-tokenizer.html

See How can I do a prefix search in ElasticSearch in addition to a generic query string? for a similar question.