Apache Solr and Arabic diacrictics

Question

I am using Drupal and I would like to index Arabic text that use Arabic diacritics extensively 'similar to accents in French or Umlauts in German".

The text is indexed but the search doesn't ignore the diacritics when searching. Does anybody know if the Arabic standard tokenizer handle Arabic diacritics?

I read somewhere that some people use AraMorph tokenizer. I am not sure if this can work with Solr 4.4.0.

I use Apache Solr integration module along with Search Api for Solr.

How can I make my search insensitive so that it ignores those diacritics?

Thanks, Mohamed

Answer 1

Adding the ICUTokenizerFactory to the text fieldtype solves the issue and the search ignores the diacritics.