0

I've been requested by a client to completely block the URLs that contain a /node/## after the hostname to prevent these locations from being indexed by search engines. My issue is that I've already defined a Page Not Found node that gets shown with error 404s.

I've already implemented a modification in the robots.txt file to include

Disallow: /node/
Disallow: /?q=node/

to prevent the web-crawler bots. However, the URLs are still directly accessible. So in this case, the client wants two different 404 actions:

  • www.clientname.com/testttt -> Drupal renders 404 page
  • www.clientname.com/node/897 -> blocked completely without returning Drupal's 404 page.

Does any module allow for this kind of functionality?

  • Just an FYI, Drupal will go to some lengths to make these URLs known to the worlds as permalinks / short URLs. In an ideal world, Globalredirect will solve this issue as it will 301 these to the alias you have defined. – mpdonadio Mar 28 '14 at 14:50
  • @MPD, but the client does not want these links redirected. They want the links blocked completely in a Apache default 403 or 404 page. – Jason Mar 28 '14 at 15:43
  • I mainly mention it as this is likely a non-issue. A spider may get to the site via the node URL, but the 301 will tell it to not index that page. I do not recall seeing any indexed node URLs on sites where Globalredirect is working properly. However, if the client requested this... – mpdonadio Mar 28 '14 at 20:07
0

What about using hook_init in a custom module?

function hook_init() {
    // check arg(2) is empty as otherwise will block node edit and delete pages
    if(substr($_SERVER['REQUEST_URI'], 0, 5) == '/node' && arg(2)=='') {
        die(); // to stop completely
        //return drupal_not_found(); // to return 404 page
        //return drupal_access_denied()(); // to return 403 page
    }
}

Saying that it would be quicker to block these pages in .htaccess...

0

I solved this with a .htaccess addition

RewriteRule ^node?$ - [F]
RewriteRule ^node/[0-9]+$ - [F]

These throw the 403 errors before Drupal catches them, which is what the client wanted.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.