How to deal with multibyte search in solr?

Question

How to tell solr to search or deal with multibyte numbers like ٦٩ , its just the number 69

in other words the user has input ٦٩ into the search box , the browser would convert it to :%D9%A6%D9%A9

solr return no result since it doesn't have the value :%D9%A6%D9%A9

how to deal with such of these query ???

solr config is the default config

Answer 1

I think you need to convert them from these characters to the number using solr.MappingCharFilterFactory https://cwiki.apache.org/confluence/display/solr/CharFilterFactories

So basically you can have a file mapping-ArabicNumbersToASCII.txt and that contains:

"٠" => "0"
"١" => "1"
"٢" => "2"
"٣" => "3"
...

Then use the it as the analyzer like so:

<analyzer>
  <charFilter class="solr.MappingCharFilterFactory" 
    mapping="mapping-ArabicNumbersToASCII.txt"/>
  <tokenizer ...>
  [...]
</analyzer>