SQL Server Free Text search: search words from a phrase in two tables

Question

I have a table with companies and one with categories. I'm using SQL Server Free Text search, and searching companies (by name and description) works fine. But now I also want to include the category table.

I want to search for something like: ABC 24 Supermarket.

Now, ABC 24 should make a match with the Name column in the company table, and Supermarket is the name of the category this company is connected to.

Right now I have something like this:

DECLARE @SearchString VARCHAR(100) = '"ABC 24 Supermarket"'
SELECT * FROM Company CO
INNER JOIN Category CA
ON CA.CategoryId = CO.CategoryId
WHERE CONTAINS((CO.[Description], CO.[Name]), @SearchString)
AND CONTAINS(CA.[Description], @SearchString)

But this of course, gives me nothing, because my search string cannot be found in either the company or the category table. Does anyone have an idea on how to do a combined search on my company and category table?

The idea of splitting strings, as suggested in Lobo's answer below, is not really an option. Because i don't know which part will be the one that should match a category and which part should be used for matching company names/descriptions. Users might just as well type in "Supermarket ABC 24".

Answer 1

Imho the right way to do this is to create an indexed view containing the primary key of the main table ('company' in your example) and a second column containing all the stuff you're actually want to search, i.e.

create View View_FreeTextHelper with schemabinding as
select CO.PrimaryKey,                            -- or whatever your PK is named
       CO.description +' '+CA.description +' '+CO.whatever as Searchtext
  from dbo.company CO join 
       dbo.category CA on CA.CategoryId = CO.CategoryId

Note the two-part form of your tables. A few restrictions arise from this, e.g. all involved tables must be in the same table space and as far as I remember, no TEXT columns are allowed in this kind of concatenation (you may cast them though).

Now create a unique index on the PrimaryKey column

create unique clustered index [View_Index] 
    on View_FreeTextHelper (PrimaryKey ASC)

Finally create the fulltext index on the view using the 'Searchtext' column as the only column to index. Of course, you may add more columns, if you e.g. wish to distinguish in searching for company name and location and, the names of the managers (you would just concatenate them in a second column).

Retrieving your data is now easy:

select tbl.RANK,
       co.* 
  from freetextTable(View_FreeTextHelper,Search,'Your searchtext here') tbl
  join company co on tbl.key=co.PrimaryKey
 order by tbl.RANK desc

You may also limit the output using select top 50 as the freetexttable clause will eventually return quite a lot of close and not so close results.

And finally, don't get confused if you cannot find thing like 'off the shelf inc.' Beware of the stop lists. These are list of words which are very common, have no semantic use (like the) and are therefore removed from the text to be indexed. In order to include them, you have to switch of the stoplist feature.

A last tipp: full text is very powerful but has a lot of features, tricks and caveats. It takes quite a bit to fully understand the techniques and get the best results you want.

Have fun.

Answer 2

If we assume that the name of columns are unique per row, then you can use below query. The following example returns all rows that contain either the phrase "ABC", "24" or "Supermarket" in each of the columns

DECLARE @SearchString nvarchar(100) = N'ABC 24 Supermarket'
SET @SearchString = REPLACE(LTRIM(RTRIM(@SearchString)), ' ', '|')
SELECT *
FROM Company CO JOIN Category CA ON CA.CategoryId = CO.CategoryId
WHERE CONTAINS(CO.[Name], @SearchString)
 AND CONTAINS(CO.[Description], @SearchString)
 AND CONTAINS(CA.[Description], @SearchString)

First of all you need to prepare a search value for the CONTAINS predicate used in the WHERE clause. In this case I replaced spaces between the words on the "|" logic operator(the bar symbol (|) may be used instead of the OR keyword to represent the OR operator.)

Answer 3

It occurs to me that while the answer I wrote previously should work fine and be reasonably efficient, processing the search items one-at-a-time and only searching within the existing search result for items after the first, that it would be faster to do it all at once, using dynamic sql.

So here is another potential solution to your problem, which I enter as a separate answer as it is unrelated to the solution I've already posted:

DECLARE @SearchString VARCHAR(100) = '"ABC 24 Supermarket"'

DECLARE @AllItems table (
  SearchItem varchar(100)
  ),
        @x    int,
        @cmd  varchar(1000),
        @wc   varchar(8000),
        @this varchar(100);

-- break up @SearchString into component search items:
select @x = charindex(' ', @SearchString);
while @x > 0 begin
  insert @AllItems (SearchItem) values (substring(@SearchString, 1, @x - 1));
  select @SearchString = substring(@searchstring, @x + 1);
  select @x = charindex(' ', @Searchstring);
end;
-- add the last item
insert @AllItems (SearchItem) values (@SearchString)

select @cmd = 'select CO.* from Company CO inner join Category CA on CO.CategoryId = CA.CategoryId WHERE';


--now process search items one-at-a-time building up a where clause to plug into @cmd:
while (select count(*) from @AllItems) > 0 begin
  select @this = min(SearchItem) from @AllItems;
  delete @AllItems where SearchItem = @this;

  select @wc = @wc +
    'AND (contains ((CO.[Description], [CO.[Name]) ''' + @this + ''') or contains (CA.[Description], ''' + @this + ''') '
end;

--ready to go:
exec (@cmd + substring(@wc, 4));  --substring removes first AND

Answer 4

I dont know whether this will constitute a GREAT answer (I tend to doubt it) but I wanted a problem to work on and I happed to pick yours, so here's my solution:

DECLARE @SearchString VARCHAR(100) = '"ABC 24 Supermarket"'

DECLARE @AllItems table (
  SearchItem varchar(100)
  ),
        @x int;

-- break up @SearchString into component search items:
select @x = charindex(' ', @SearchString);
while @x > 0 begin
  insert @AllItems (SearchItem) values (substring(@SearchString, 1, @x - 1));
  select @SearchString = substring(@searchstring, @x + 1);
  select @x = charindex(' ', @Searchstring);
end;
-- add the last item
insert @AllItems (SearchItem) values (@SearchString)


DECLARE @this varchar(100),  -- = current search item
        @found table (       -- table to contain rows matching the current search item
          ID int
          ),
        @usable table (      -- table to contain rows matching all search items
          ID int             --   already tested
          );

--now process search items one-at-a-time
while (select count(*) from @AllItems) > 0 begin
  select @this = min(SearchItem) from @AllItems;
  delete @AllItems where SearchItem = @this;

  if (select count(*) from @usable) = 0 begin  --first search item
    --for the first item, just find the companies matching this item, in either the 
    --company name or description or category description columns:
    insert @found (ID)
    select CO.CompanyID
    from Company CO inner join Category CA on CO.CategoryID = CA.CategoryID
    where contains ((CO.[Description], [CO.[Name]) @this)
      or contains (CA.[Description], @this)
  end 
  else begin                                   --other search items
    -- for subsequent items, its got to match with the company name or description
    -- or category description as above - BUT it's also got to be a company we 
    -- already identified when processing the previous term
    insert @found (ID)
    select CO.CompanyID
    from Company CO inner join Category CA on CO.CategoryID = CA.CategoryID inner join @usable U on CO.CompanyID = U.ID
    where contains ((CO.[Description], [CO.[Name]) @this)
      or contains (CA.[Description], @this)
  end 

  --now clear out and re-populate the usable companies table ready for processing the
  --next search item
  delete @usable;
  insert @usable (ID)
  select ID
  from @found;

  --and clear out the current matches table, ready for the next search item
  delete @found;
end;


--whatever is in @usable now, is a match with all the component search items, so:
select CO.*
from Company CO inner join Category CA on CO.CategoryId = CA.CategoryId inner join @usable U on CO.CompanyID = U.ID;

Answer 5

The idea here is that we are going to parse the string into different variables then search the other variables for a match.

DECLARE @SearchString VARCHAR(100) = '"ABC 24 Supermarket"', @A varchar(100), @B varchar(100), @C varchar(100),@index int

set @A = Substring(@searchString, 1, PATINDEX('% %', @searchString) -1)
set @index = PATINDEX('% %', @searchString) + 1
set @B = Substring(@searchString, @index, PATINDEX('% %', @substring(@searchstring, @index, 100)) -1)
Set @index = PATINDEX('% %', @substring(@searchstring, @index, 100)) + 1
set @C = Substring(@searchString, @index, PATINDEX('% %', @substring(@searchstring, @index, 100)) -1)

SELECT * FROM Company CO
INNER JOIN Category CA
  ON CA.CategoryId = CO.CategoryId
WHERE CO.[Description] like @A
  or CO.[Description] like @B
  or CO.[Description] like @c
  or CO.[Name] like @A
  or CO.[Name] like @B
  or CO.[Name] like @C
  or CA.[Description] like @A
  or CA.[Description] like @B
  or CA.[Description] like @C

This code looks ugly to me but it should accomplish the requirements for the user entering up to 3 items to search on. Anyone have suggestions on cleaning it up?

Answer 6

this should work.

DECLARE @SearchString VARCHAR(100) = '"ABC 24 Supermarket"'
set @SearchString = replace(@SearchString,' ','" or "')

SELECT * FROM Company CO
INNER JOIN Category CA
ON CA.CategoryId = CO.CategoryId
WHERE CONTAINS((CO.[Description], CO.[Name]), @SearchString)
AND CONTAINS(CA.[Description], @SearchString)

hope helps a bit

Answer 7

Why don't you just reverse your logic? Your example was trying to find the search string within your field values, but what you really want to do is find your field values within your search string, no?

    DECLARE @SearchString VARCHAR(100) = '"ABC 24 Supermarket"'
SELECT * FROM Company CO
INNER JOIN Category CA
ON CA.CategoryId = CO.CategoryId
WHERE (CONTAINS(@SearchString, CO.[Description]) OR CONTAINS(@SearchString, CO.[Name]))
AND CONTAINS(@SearchString, CA.[Description])

Answer 8

( I don't have sql-server installation to try CONTAINS. You can replace the column LIKE '%string%' with CONTAINS(column, 'string') and try.)

See all queries here.

---

Another update - After reading the other answers and the manual, it seems that you don't need parenthesized values in the contains string unlike I expected. So this should work too - (you may even try ' | ' instead of ' OR '

SELECT CO.name, CA.description FROM company CO
         INNER JOIN category CA
             ON CA.CategoryId = CO.CategoryId
WHERE CONTAINS((CO.name,CO.description), REPLACE('ABC 25 SuperMarket', ' ', ' OR '))
      AND
      CONTAINS(CA.description, REPLACE('ABC 25 SuperMarket', ' ', ' OR '))  

If it complains about syntax error near replace, you can create a search string DECLARE @SearchString varchar(MAX) = REPLACE('ABC 25 SuperMarket',' ', ' OR ') and then use it in place of replace(......) as second argument.

---

Update as per your modified question -

Firstly, you should be moving the logic to application level if possible. I think, it is too much to handle it here. I have come up with this query, but note that this will split each word and search for it in both name and description so you will end up getting a few more results than what you might think. For e.g. this will return all Supermarket which have either ABC or 24 in their name compared to returning only one Supermarket with name ABC 24 in my previous query. This should actually help you out because, as per you, user might just type "ABC Supermarket 24" or "24 ABC Supermarket" or ...

DECLARE @SearchString varchar(MAX) = 'ABC 24 SuperMarket'
DECLARE @separator varchar(MAX) = ' '
DECLARE @Like1 varchar(MAX) = 'CO.name LIKE'
DECLARE @Like2 varchar(MAX) = 'CA.description LIKE'

DECLARE @WHERE1 varchar(MAX) = '( ' + @Like1 + ' ''%' + 
           REPLACE(@SearchString,@separator,'%'' OR ' + @Like1 + ' ''%')+'%'')'

DECLARE @WHERE2 varchar(MAX) = '( ' + @Like2 + ' ''%' + 
           REPLACE(@SearchString,@separator,'%'' OR ' + @Like2 + ' ''%')+'%'')'

DECLARE @QueryString varchar(MAX) =
           CONCAT('SELECT CO.name, CA.description FROM company CO
                       INNER JOIN category CA
                             ON CA.CategoryId = CO.CategoryId
                    WHERE ', @WHERE1, ' AND ', @WHERE2)
exec(@QueryString);

If you output @WHERE1 you should see

( CO.name LIKE '%ABC%' OR CO.name LIKE '%25%' OR CO.name LIKE '%SuperMarket%')

As I said before you might want to try using CONTAINS with parenthesized values like

DECLARE @SearchString varchar(MAX) = 'ABC 25 SuperMarket'
DECLARE @separator varchar(MAX) = ' '

DECLARE @WHEREString varchar(MAX) = '''"' + 
        REPLACE(@SearchString, @separator, '" OR "')+'"'''

SELECT CO.name, CA.description FROM company CO
        INNER JOIN category CA
           ON CA.CategoryId = CO.CategoryId
WHERE CONTAINS((CO.name,CO.description), @WHEREString)
      AND 
      CONTAINS(CA.description, @WHEREString)

If you output @WHEREString you should see

'"ABC" OR "25" OR "SuperMarket"'

---

Previous answer:

This will assumes that the word after the last space is the description and rest is the `name.

You can split the search string and use them as shown below. This query is using like as I don't have a sql-server installation.

DECLARE @SearchString VARCHAR(100) = 'ABC 24 Supermarket'
DECLARE @searchLength int = len(@SearchString)
DECLARE @searchReverse VARCHAR(100) = reverse(@SearchString)

SELECT CO.name, CA.description FROM company CO
        INNER JOIN category CA
            ON CA.CategoryId = CO.CategoryId
WHERE CO.name LIKE concat( '%', SUBSTRING(@SearchString,0,@searchLength-charindex(' ',@searchReverse)+1), '%')
     AND 
     CA.description LIKE concat( '%', SUBSTRING(@SearchString,@searchLength-charindex(' ',@searchReverse)+2,@searchLength), '%')

This should work. Please note that the where clause is using AND instead of OR.

DECLARE @SearchString VARCHAR(100) = 'ABC 24 Supermarket'
DECLARE @searchLength int = len(@SearchString)
DECLARE @searchReverse VARCHAR(100) = reverse(@SearchString)
DECLARE @company VARCHAR(100) = SUBSTRING(@SearchString,0,@searchLength-charindex(' ',@searchReverse)+1)
DECLARE @category VARCHAR(100) = SUBSTRING(@SearchString,@searchLength-charindex(' ',@searchReverse)+2,@searchLength)
    
SELECT CO.name, CA.description FROM company CO
        INNER JOIN category CA
           ON CA.CategoryId = CO.CategoryId
WHERE CONTAINS((CO.name, CO.description), @company)
      AND
      CONTAINS(CA.description , @category)

Answer 9

you can use FREETEXT instead of CONTAIN.

DECLARE @SearchString VARCHAR(100) = '"ABC 24 Supermarket"'
SELECT * FROM Company CO
INNER JOIN Category CA
ON CA.CategoryId = CO.CategoryId
WHERE FREETEXT((CO.[Description], CO.[Name]), @SearchString)
OR FREETEXT(CA.[Description], @SearchString)