1

I have a site that's in English, but there's a fair amount of French text on it. So, there are some special characters like è, é, á, etc.

I don't want to worry about those diacritics at all when searching (using solr 4), so that, say "any e gives you every e"

Right now, I have it so if you search for non-diacritics, you get all the related diacritics (search for "siecle", and get "siècle"), but if you search using diacritics it won't find anything (search for "siècle" returns nothing.)

I've been up and down between playing around with mapping-ISOLatin1Accent.txt and using:

<charFilter class="solr.MappingCharFilterFactory" mapping="mapping-ISOLatin1Accent.txt"/>

and

<filter class="solr.ASCIIFoldingFilterFactory" preserveOriginal="false"/>

But nothing I've tried seems to have any change to this. I assume I have to add this to both AND in schema.xml, but I'm not sure which fieldtypes to do that in. There are fieldtypes of:

  • text_ws
    • "text field that only splits on whitespace for exact matching of words"
  • text
    • "A text field that uses WordDelimiterFilter to enable splitting and matching of words..."
  • text_und
    • "An unstemmed text field - good if one does not know the language of the field"

I've tried every combination I can think of, and cleared/rebuilt the index each time, but nothing. A search including diacritics always fails.

After looking through the admin site for Apache, it looks like the ASCIIFoldingFiltorFactory is being added to the index analyzer, but NOT the query analyzer for the fieldtypes of text, but I don't know why; I added it the same way to both analyzers.

0

So, I think it was a mixture of problems/issues that arose, but they were more broad/were somewhat indirectly related to the symptoms. I'll mention the steps to fix it/figure it out:

  1. On the web amin interface, I selected the core from the dropdown, then "Schema Browser" below it.
  2. From there, I could tell which ACTUAL filters/tokenizers were getting loaded. The ASCIIFoldingFilterFactory was listed on the index analyzer for the text fieldtype, but not the query analyzer.
  3. When I attempted to reload the core, it got an error, and failed. It said something about one of the diacritics already being loaded (it was talking about the mapping-ASCIILatinwhatever.txt file that I'd tried earlier)
  4. I cleared that file, since I was trying to use a different filter, and the core was able to reload, reindex, and everything's working great now (and the filter is on the query analyzer.)

So, anyone else dealing with this issue, in suhort: Check your "Schema Browser" in the web gui for Solr on the core you're having trouble with!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.