Showing results for 
Search instead for 
Did you mean: 
Create Post

Logs, Logs, and More Logs

Level 9


Four score and one post ago, we talked about Baltimore’s beleaguered IT department, which is in the throes of a ransomware-related recovery.

Complicating the recovery mission is the fact that the city’s IT team didn't know when the systems were compromised initially. They knew when the systems went offline, but not if the systems were infected earlier. The IT team can’t go back and check a compromised system’s logs because ransomware rendered the infected computers inaccessible.

Anyone who has worked in IT operations knows logs can contain a wealth of valuable information. Logs are usually the first place you go to troubleshoot or detect problems. Logs are where you can find clues of security events. Commonly, though, you can end up having to sift through a lot of data to find the information you need.

In any ransomware or security attack, a centralized logging server or Syslog is an invaluable resource to trace back and correlate events across a plethora of servers and network devices. Aggregation and correlation are jobs for a SIEM.

All About Those SIEMs

SIEM is mostly mentioned as an acronym, not its extended form of Security Information and Event Management tool. SIEMs serve an essential role in security threat detection and commonly make up a valuable part of an organization’s defense-in-depth strategy.

SIEM tools also form the basis for many regulatory auditing requirements for PCI-DSS, HIPAA, and NIST security checks, as well as aid with threat detection.

In a video recording of a session at RSA 2018, a presenter asked the audience who was happy with their current SIEM. When no hands went up, the presenter quipped that maybe the happy people were in the next room.

If I were in that room, I wouldn’t raise my hand either. On a previous contract, our SIEM tool consumed terabytes upon terabytes of space. When it came to time to pull information, the application was slow and unresponsive. Checking the logs ourselves was a more efficient use of time. So, why did we do this? Our SIEM was a compliance checkbox.

Extending SIEMs With UEBA and SOAR

SIEMs are much more than compliance checkboxes. User and Entity Behavior Analytics (UEBA) and Security Orchestration, Automation, and Response (SOAR), when bundled with SIEMs, offer additional features to extend security event management features.

UEBAs look for normal and abnormal behavior for both users and entities to improve visibility across an organization. By using advanced policies and machine learning, UEBAs improve visibility to help protect against insider threats. However, like SIEMs, UEBAs may require fine-tuning to weed out the false positives.

SOARs, on the other hand, are designed to automate and respond to low-level security events or threats. SOARs can provide similar functionality to an Intrusion Detection System (IDS) or Intrusion Prevent System (IPS), without the manual intervention.


At the end of the day, SIEMs, SOARs, UEBAs, and other security tools can be challenging to configure and manage. It makes sense for organizations to begin outsourcing part of this responsibility. Also, you could argue that applications reliant on machine learning belong in cloud-like environments, where you could build large data lakes for additional analytics.

In traditional SIEMs, feeding in more information probably won’t result in a better experience. Without dedicated security analysts to fine-tune the data collected, many organizations struggle with unwieldy SIEMs. While it’s easy to blame the tool, in many cases, the people and processes contribute to the implementation woes. 

Level 11

Logs are great, but it's important to find the balance, as otherwise you can have too much information, which can confuse or obfuscate issues.

Level 13

Thanks for the article.

Level 14

Thanks for the article.  And I agree with janobi​.  Any data isn't good data - it has to be relevant. 

Level 16

Thanks for the write up.

Level 15

Logs are a great tool for tracing what happened.  But, in relation to SIEM's there is the need to correlate the Log data with packet captures.  I have spent countless hours gleaming through logs both inside the SIEM as well as a syslog server to determine an issue, only to be halted by the fact that a packet capture is required to determine the root cause.

Thanks for the post.

Level 13

Good post, thanks.  I keep hoping we're going to find a centralized logging solution that checks all the main boxes in terms of what we need in a SIEM but so far it hasn't happened. So far outsourcing some of it has wored really well and is a nice complement to what we have on prem.


Nice write up

Level 11

Thanks for the article.


Our SIEM was a compliance checkbox.

YUP. All hail compliance! Compliance allows us to do a lot of stuff, spend a lot of money, and deliver a lot of presentations without actually improving the security posture of an organization.

Level 20

Lot's of logs 4 sure!  Using SEM instead of splunk because it's much more cost effective.  I'm glad SEM aka LEM is being rewritten in HTML5!

Level 12

I wonder how detailed the logs are. Logs that aren't quite verbose will not show when software was added to a system. Logs that are this verbose will  be difficult to review when you need to find an infection because they will be so large.

Level 12

logs management is a critical things for the company i work for because with opensource/free solutions we tried to setup the huhe amount of logs send the web application in crash not recovering anymore from this state. yumdarling

About the Author
Becky Elliott, a Baltimore native, has worked in Information Technology for over 20+ years, mostly as a Government Contractor. In recent years, she has leaned into Tech Community as an NetApp A-Team Advocate, Tech Field Day Delegate, and aspiring “extra credit kid”. She holds a number of certifications including CISSP, Linux+, NetApp Certified Implementation Engineer - SAN.