0

We have a search page that uses facets and recently our website has been going up and down. When looking at the logs I noticed that bots (Bing, Ahrefs, etc..) are hitting these facetted pages a lot. Since the facetted pages are not cached I think it is causing this issue.

I did add Search-specific caching but that did not help.

I have seen these related issues with caching and facets:

Has anyone else ran into an issue like this? For now I blocked all bots from accessing the facetted urls (so they will still be able to see the main search page).

0

For example, my facet page path starts with /library. I created a Firewall rule in Cloudflare to block anyone visiting a page that contains /library/ so a facet result page like /library/xxx/bbb/ccc and is a known bot will be blocked.

enter image description here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.