1

I've been looking at my site logs recently. I have the limit of logs set to 10,000. The problem is, due to the amount of traffic we get, that number will only represent, at best, half a day of logs. Most of these are ether from access denied (I'm guessing spam bots trying to create nodes) or page not founds. The rest are items from custom modules I've built that I use to monitor how those modules are functioning (Like a notification that a queue is being run, or nodes are being created, etc.).

I've toyed with the idea of bumping the log messages to keep up to 100,000 or 1,000,000, but I'm not sure if this will cause performance issues or if it will drastically increase the size of my db.

Ultimately though, that would only really be a band-aid at best. As it would be nice to be able to keep a longer track of error messages, say weeks or months, so that I can further analyze what my site is doing, what areas need to be improved, etc. Is there any way to create a back up of log files so I don't loose them forever once they hit that ceiling?

5

I believe it's a best practice to not enable the core Logging module for Production Drupal websites (eg, outside of your development environment) and to setup server (host) based logging -- or offload the logging to a remote service.

See the Syslog documentation page. Which is what I mean by "host based logging".

The Syslog module logs events by sending messages to the logging facility of your web server's operating system. Syslog is an operating system administrative logging tool that provides valuable information for use in system management and security auditing. Most suited to medium and large sites, Syslog provides filtering tools that allow messages to be routed by type and severity.

It is not suitable for shared hosting environments.

It might not be as user friendly as Database Logging but will allow you to see logs and troubleshoot if your site is not accessible. Because the Database logging module writes logs to the database it can slow down the website. By using Syslog you can improve the performance of the site.

An example of a remote log is Redis Logging, there are numerous other 3rd party logging modules.

EDIT

A common practice for webservers is to Rotate Logfiles which builds an archive of compressed logfiles after your logfile has accrued to a certain size or after X days. By using Syslog you can configure your webserver to then rotate logfiles.

3
  • 1
    Definitely a good idea to keep logging out of the db if at all possible - a little while ago on a high-traffic site I maintain a remote filesystem went AWOL for a while which resulted in a ton of 404 errors being logged for every request. Sheer weight of traffic combined with the extra db load from the watchdog made the whole site basically unresponsive until the dblog module was disabled manually. These things don't happen often, but when they do it's doubly annoying if you could've prevented it!
    – Clive
    Sep 24 '14 at 17:31
  • Very good point, I'll look into getting this set up. Thanks for the help guys!
    – Jance
    Sep 24 '14 at 17:57
  • You can use a service like Splunk to offload your logs for later use. Sep 25 '14 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.