Troubleshoot Java Lucene ignoring Field
Question
We're currently using Lucene 2.1.0 for our site search and we've hit a difficult problem: one of our index fields is being ignored during a targeted search. Here is the code for adding the field to a document in our index:
// Add market_local to index
contactDocument.add(
new Field(
"market_local"
, StringUtils.objectToString(
currClip.get(
"market_local"
)
)
, Field.Store.YES
, Field.Index.UN_TOKENIZED
)
);
Running a query ( * ) against the index will return the following results:
Result 1:
title: Foo Bar
market_local: Local
Result 2:
title: Bar Foo
market_local: National
Running a targeted query:
+( market_local:Local )
won't find any results.
I realize this is a highly specific question, I'm just trying to get information on where to start debugging this issue, as I'm a Lucene newbie.
UPDATE
Installed Luke, checking out latest index... the Field market_local is available in searches, so if I execute something like:
market_local:Local
The search works correctly (in Luke). I'm going over our Analyzer code now, is there any way I could chalk this issue up to the fact that our search application is using Lucene 2.1.0 and the latest version of Luke is using 2.3.0?
Solution
For debugging Lucene, the best tool to use is Luke, which lets you poke around in the index itself to see what got indexed, carry out searches, etc. I recommend downloading it, pointing it at your index, and seeing what's in there.
OTHER TIPS
The section on "Why am I getting no hits?" in the Lucene FAQ has some suggestions you might find useful. You're using Field.Index.UN_TOKENIZED, so no Analyzer will be used for indexing (I think). If you're using an Analyzer when you're searching then that might be the root of your problem - the indexing and searching Analyzers should be the same to make sure you get the right hits.
Another simple thing to do would be to use a debugger or logging statement to check the value of
StringUtils.objectToString(currClip.get("market_local"))
to make sure it is what you think it is.
Luke is bundled with Lucene, but you can tell Luke to use another version of Lucene. Say "lucene-core-2.1.0.jar" contains Lucene 2.1.0 that you want to use and "luke.jar" contains Luke with Lucene 2.3.0. Then you can start Luke with the following command.
java -classpath lucene-core-2.1.0.jar;luke.jar org.getopt.luke.Luke
(The trick is to put your version of Lucene before Luke on the classpath. Also, This is on Windows. On Unix, replace ";" with ":".)
As you can check in Luke,
+( market_local:Local )
gets rewritten to
market_local:Local
if the rewrite(IndexReader) method of the Query object is called. The two queries should be equivalent so there might be a bug in 2.1. If you have to use 2.1, you can try to manually call that method before passing the Query object to the IndexSearcher.