How to noindex comment/reply in Drupal 7

Question

I use Drupal 7, and I am trying to use the "noindex" meta tag to prevent http://example.com/comment/reply/43/738 from being indexed by search engines.

The comment/reply page is being indexed, even though I've used the following role in robots.txt:

Disallow: /comment/reply/

In which template file will I need to incorporate "noindex" meta tag for /comment/reply/ pages?

Answer 1

You can use any of this code in robots.txt:

User-agent: *
Disallow: /*comment

This will neglect every URL that contains comment.

You can also use the following, to neglect every URL containing /comment.

User-agent: *
Disallow: /comment/reply

User-agent: *
Disallow: /comment

After you do this, check whether it is working or not by using the Google Robots.txt checker.

Answer 2

In your theme's template.php, or in page.tpl.php, you can check the page's url to see if the page is a comment page, then add code that will insert the meta tag.

You can do this in the template.php function YOURTHEME_preprocess_html(), or insert it somewhere near the top of page.tpl.php. The code to add would be something like this:

    <?php
      $element = array(
        '#tag' => 'meta', 
        '#attributes' => array(
          'property' => 'robots',
          'content' => 'noindex',
        ),
      );
    drupal_add_html_head($element, 'robots');
?>

See drupal_add_html_head().

Answer 3

Update: the following works, it prints <meta name="robots" content="noindex,follow" /> in the head section of all pages accessed via a path starting with comment. This is to work around the fact that a page like comment/3 internally carries node/nid as arguments, as jmarkel pointed out above.

<?php
function metarobots_comment_help() {
  $url_components = explode('/', request_uri());
  if ($url_components[1] == 'comment') {
    $elements = array(
      '#tag' => 'meta',
      '#attributes' => array(
        'name' => 'robots',
        'content' => 'noindex,follow',
      ),
    );
   drupal_add_html_head($elements, 'robots');
  }
}

With the aid of a helpful comment on the arg function.

Hopefully this will eventually be addressed by Meta Tags module - there's a feature request, but it's not yet clear if the module author wants to support this.

---

[My previous comment] I just wanted to add that using robots.txt is not the answer - as you already noticed, links keep appearing in the search results despite using Disallow: /comment. This is expected, since robots.txt tells the bot not to crawl those pages, but it doesn't tell Google not to index it. As explained on SEOmoz Robots.txt and Meta Robots Best Practices for Search Engine Optimization:

In most cases, meta robots with parameters "noindex, follow" should be employed as a way to to restrict crawling or indexation.

Block with Robots.txt - This tells the engines to not crawl the given URL but tells them that they may keep the page in the index and display it in in results.

Block with Meta NoIndex - This tells engines they can visit but they are not allowed to display the URL in results. (This is the recommended method) So the noindex meta tag you're after is indeed what you need.