Logs 101 Part 2 - Logs: So Many Different Types

phil3 over 11 years ago 2 minute read time

This is the second post in a five-part series uncovering the mysteries of log management. In Part 1 - What Are Logs?, we discussed how logs can be useful for troubleshooting, compliance reporting, and even proactive problem remediation. In this post, we'll look at how utilizing logs can sometimes be challenging given their various types and formats.

The Challenge of Viewing Logs of Various Types

One of the challenges of making use of the valuable logs in your environment is that there are so many different types. To complicate the matter even further, each type has its own method for collecting and storing the log data. We'll look a little deeper into this question of log collection in Part 2 of this series. For now, let's just look at the four most common types of log files:

Windows Event Logs - The Windows Event Logs are what most IT professionals are familiar with from a troubleshooting perspective. After all, what breaks more than Windows? (Just kidding.) But this is not why Windows Event Logs warrant their own category in this discussion. Windows stores its logs in a proprietary format that is unique compared to each of the other log types. The most common way to access Windows Event Logs is the Event Viewer MMC snap-in, and Windows logs events in a variety of categories, including Application, Security, and System.
Text - Text logs are the most prevalent if only because there are so many ways to store and transmit text. Text logs include logs that are transmitted from network devices using the syslog protocol, logs that are stored in various text formats in the related application's installation directory, and logs for all Linux operating systems. It's important to note here that not all "text" logs are human-readable, much less accessible via a text editor.
SNMP - Network devices and computer systems use simple network management protocol (SNMP) to store and transmit their state and status information. SNMP logging is most common in network devices such as routers and switches, but it's also utilized by applications, such as McAfee EPO to manage network security.
Database - Logging directly to a SQL (or similar) database seems to be a new trend in the logging world. This gives the consumer of the logs a lot more flexibility when querying, viewing, and archiving logs, and it also adds a layer of security by restricting access to only those who can authenticate to that database.

In Logging, Type ≠ Format

One footnote to this "type" discussion is that the log type does not necessarily equate to the log format. For example, text logs may be in syslog format to be transmitted using the syslog protocol, but they could also be in Snort, W3C, or some proprietary format. Similarly, SNMP logs depend on their related management information bases (MIBs), while database logs all have their own schema. I wouldn't count on all of the Windows Event Logs to be in the same format either.

Finally, as you consider implementing a log management procedure or solution in your environment, keep in mind that logs are not always human readable. Sure, with practice, you should be able to learn the different "languages" your systems and devices use for logging, but the scope of that complexity is pretty broad. But before you can read the logs, you have to get them. For more information about that, stay tuned for the next post in this series: