My site's drupal version is 6.19. There are always logs like the following.
the recent log entries shows
type: page not found location:http://www.example.com/display.html referrer: message: 500.shtml severity: warning hostname: 184.108.40.206 or others (maybe search engine robot IP)
The URL and the content is right, but when the anonymous user accesses it, it returns the "Page not found" page. Anonymous users have the permissions to access content.
The cache_page table contains the following header for http://www.example.com/display.html:
Content-Type: text/html; charset=utf-8 HTTP/1.1 404 Not Found
Why is the page being cached as "404 Not Found", when it actually exists?
When I access http://www.example.com/display.html as an anonymous user, the site shows "page not found"; when I clear the cache, anonymous users can access the page.
How can I correct the cached content generated by search engine bots?