October 1, 2015

Proactive Server Log Management Puts You in Front of Problems

Written by Tom

keyword icon Logging

Hey, you're busy. You don't have time to read log files. Besides, there are so many of them, and most don't contain any information worth the time it takes to read them. In fact, the only time you read any log files is when you're trying to figure out what just went wrong.

Now reverse that scenario: Your log files are scanned continually to spot and alert you to any anomalies that could spell trouble. This lets you fix problems while they're little and relatively easy to address. It also frees up some of your valuable time for all those other tasks commanding your attention.

The primary benefit of a proactive approach to log management is predictive analysis: identify anomalies and behaviors that in the past have been precursors of trouble. The first challenge in creating a proactive log-analysis procedure is collecting the log files themselves. This task becomes more difficult as the systems we're managing are distributed more widely.

In a July 2015 article, ComputerWeekly's Bob Tarzey lists some of the many potential sources of log files in modern networks:

  1. Servers (physical and virtual)
  2. Routers
  3. Load balancers
  4. Application delivery controllers
  5. Firewalls
  6. User devices
  7. Cloud services

Tarzey point out that while security has traditionally been cited as the primary reason for adopting a real-time logging and alert system, two other important areas benefit as well: general IT management, and business intelligence. Use cases in the first category include identifying inactive (ghost) and rogue virtual servers in cloud environments; and ensuring applications have all the resources they need to operate effectively.

Reactive alerts may take hours to respond to, while proactive alerts prevent downtime and data loss, and predictive alerts notify you of potential problems before they occur. Source: APM Digest

Applications of log data to business intelligence include the ability to correlate call-center volumes with customer behavior; and measurement of key performance indicators (KPI) to monitor a web service's transaction times.

Planning your integrated log-management strategy

TechTarget's Bill Kleyman outlines a seven-step log-management program:

  1. Create an audit trail to allow forensic engineers to correlate anomalous behaviors with specific users;
  2. Track all intrusion attempts and activate alerts when unauthorized accesses occur;
  3. Activate alerts whenever such activities are detected and respond quickly to contain the intrusion and minimize damage;
  4. Establish baseline log values and alert quickly when the values exceed specified ranges of normal activity;
  5. Ensure that operations staff can access and monitor system alerts in real time to minimize downtime and prevent data loss;
  6. Leverage the network and system baselines to establish values that can be used to plan future systems;
  7. Create living log workbooks that can be used to coordinate management of individual systems throughout the organization's data infrastructure.

So invest now in proactive log management, and avoid problems in the future.