0

We're working with an SEO specialist who is finding that our underlying /node/### page URLs are competing with our aliases and bringing our page ranking, etcetera down - because they're perceived as duplicate content. (Drupal 8)

I'm looking at anything that is out there, but not sure if any of these cover what I need. Is there a simple way to set all urls containing /node/* to noindex, nofollow? Maybe a way in robots.txt?

Would simply adding this to robots.txt work?

Disallow: /node/*

As a side question, what are the ramifications of making such a change? Any issues I need to be aware of, protect against?

Or maybe there's another way people have solved this problem?

Improve this question

1 Answer 1

Reset to default
2

For Drupal8 and 9 what you need is simply this module: https://www.drupal.org/project/redirect. It will redirect all /node/* to its alias. For Drupal7: https://www.drupal.org/project/globalredirect

Improve this answer
6
  • I already have the redirect module in place and active. Maybe there's a setting somewhere that needs to change? I don't want just redirects. I want to prevent indexing of /node/### altogether with a 'noindex' - According to the SEO specialist, Google is still perceiving these as duplicate content
    – turpentyne
    Jun 24, 2020 at 18:22
  • @turpentyne – Yeah it's a simple setting. Done. Then look at it from Googlebots point of view: Somehow the node/ID URLs got indexed (have you removed them from the sitemap and prevent they appear in the metatags as short URL as well?) and now tries to recrawl one of these URLs, gets a 301 redirect, doesn't index, comes to path alias, check if its allowed to index, then does that. On that path alias page try to remove all references to node/ID. Sometimes they are in the metatags, sometimes in the language switcher, sometimes in the content.
    – leymannx
    Jun 24, 2020 at 18:27
  • @turpentyne – Sometimes it takes a while until already indexed node/ID URLs get unindexed and replaced by the URL they are redirecting to.
    – leymannx
    Jun 24, 2020 at 18:31
  • @leymannx with a little further investigation, I think I might know what's happened. It's an older site, rebuilt in D8 2 yrs ago. some older nodes were imported before adding redirect. They have aliases, but aren't redirecting. Maybe I have to manually add redirects, or is there a way to batch all urls with aliases to capture the older ones? (anything added since Redirect was installed, creates the redirect automatically)
    – turpentyne
    Jun 24, 2020 at 18:40
  • @turpentyne – Sounds like doing that manually or happy coding! Get a dump from live set up page locally then gamble around a bit with hook_update_N or maybe a custom form and a batch process if there are that many. And then simply drop and reimport the dump while testing until you know how it's done and then deploy the update hook or custom form to live. 🤓
    – leymannx
    Jun 24, 2020 at 18:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.