2

I configured a search box for a drupal site and am a bit confused on how to get a search working with partial matches and a sensible autocomplete.

Some basic details:

  • Drupal: 9.3.9
  • Modules: Search API, Search API Autocomplete, Search API Solr
  • Solr: 8.11.1

The search searches through different fields from several types of content. At first I set the field types to "Fulltext". Then I noticed that only complete words are matched. So searching for "idea" would not find pages containing "ideas". That worked when I changed all field types to "Fulltext ngram". But: Now the autocomplete suggests all the partials as well. So entering "super" would suggest "superh", "superhe" and "superhero". That's not really useful for the site's visitors.

What I already tried: The autocomplete uses the suggester "Retrieve from server". I tried to enable the additional module "search_api_solr_autocomplete". That adds three more suggesters, "Solr Spellcheck", "Solr Suggester" and "Solr Terms". The spellcheck suggester did not yield any autocomplete suggestions. Neither did the Solr Suggester. Solr Terms does the same as "Retrieve from server", it suggests all intermediate steps between the search word and the finished term, letter for letter.

What I am looking for is a clue how to configure this so that the search matches tokens with partial searches, but that only complete tokens are used for the autocomplete feature. Maybe a configuration step is missing?

4
  • Is your search block created using views with an exposed filter using "Fulltext search" field? Apr 5 at 15:33
  • Yes, I use a view with a fulltext search filter. Apr 5 at 16:23
  • You may have to create two indexes and then use this drupal.org/project/fac on your search input field. This would search the index without ngram for the autocomplete but when you press search/submit it uses the view. Apr 5 at 18:20
  • Sorry for the lack of feedback. I spent so much frustrating time on this issue, that I needed some time away from it. My current solution is that I have a new field which is a concatenation of all the other fields' contents. This field is the source for the autocomplete. This seems like the right approach, but is not working in practice. I think, I struggle with selecting the right field type for the new field. May 9 at 8:31

1 Answer 1

1

I couldn't get this to work previously, but after upgrading to Drupal 9.4 it now works. What I did was this:

  • I added a new field "autocomplete_tokens" as an "aggregated_field". I put all the text fields in there that were part of the search, especially the rendered HTML and the file contents.
  • As a field type I chose "Fulltext Tokens", which is defined as "Full text field without any processing like stemming or stop word filters, just unmodified tokens from the text separated by white spaces". This way, all the words in the input stay exactly as they are. No stemming, no breaking up ito ngrams.
  • In the "autocomplete" tab of my index, I clicked on "Edit" and chose the suggester "Retrieve from server". Under "Configure suggester Retrieve from server > Override used fields" I chose the new field "autocomplete_tokens". This way, the autocomplete feature got its suggestions only from my new field, not from all the other stemmed and ngrammed fields.

And that's it, this time it worked like a charm. Reindex all contents (since a new field was added and needs to be present in the index), and the autocomplete now behaves as expected. I can only guess that I did something stupidly wrong or that something changed with Drupal 9.4.

This solution has some caveats, of course. First, when I add a new field to be part of the search (e.g. for a new content type), I have to remember to add it to the autocomplete token concatenation field as well. This can easily be forgotten. On the other hand, I also have to add the new field to the view and maybe other places, so this needs to be tested anyway.

I also found that this datatype does not remove punctuation from the tokens. So if my content contains e.g. "unbowed, unbent, unbroken", the autocomplete suggestion for "unb" is "unbowed," and "unbent," along with "unbroken" (or "unbroken.", if it ends as a sentence). This can maybe dealed with using a different field type or adding a special field type to the schema.xml. Since I wanted to close this issue fast, I added a filter instead to my local search module:

function my_module_search_api_autocomplete_suggestions_alter(array &$suggestions, array $alter_params) {
  $allNewSuggestions = [];
  foreach ($suggestions as $index => $suggestion) {
    // Remove punctuation from the end of autocomplete suggestions.
    $newSuggestionSuffix = rtrim($suggestion->getSuggestionSuffix(), ".!?,-'\"");
    $suggestion->setSuggestionSuffix($newSuggestionSuffix);

    // Now remove duplicates that occur if two different input suggestions are
    // reduced to the same token when removing the punctuation.
    $newSuggestion = $suggestion->getSuggestionPrefix() . $newSuggestionSuffix;
    if (in_array($newSuggestion, $allNewSuggestions)) {
      unset($suggestions[$index]);
      continue;
    }
    $allNewSuggestions[] = $newSuggestion;
  }
}

This removes all punctuation from the end of the suggestions and removes duplicates (if a word was suggested one with a trailing comma and once with a trailing full stop, for example). Now we might have fewer than the configured number of sugestions, but I can live with that.

I also notice that all the suggestions are in lower-case, which looks a bit strange for German nouns. But for now I can also live with that.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.