great question. I'm about to face the same. Also, what are the system recommendations for proc/RAM for more data if any?
Prior to LEM version 5.6, the limit was 1TB. With version 5.6, the 1TB limit has been removed. The new limit is 2.2TB, which is the next most common barrier is 2.2TB, based on virtual infrastructure capabilities to address a single disk. This is according to http://thwack.solarwinds.com/community/solarwinds-community/product-blog/blog?start=45.
Dang, I new there was a blog that covered that but I wasn't able to find it before asking the question here. Thanks for pointing that out!
So, after thinking about this I have a follow-up question...
What if you need to store more than 2.2TB worth of data to meet a data retention requirement?
Here's an example related to PCI. PCI requires 90 days of logs online, with 365 available offline. In this scenario, make sure that you have a DB size that allows the 90 days to be online, plus a buffer, just to be sure. Then save your backups to have the 365 days offline available. In your question, you didn't elaborate on what your data retention requirements are, but this is an example.
If you have your DB size maxed out and still don't have enough room for your *online* data retention requirement, then you'll have to look at a couple of things. I would recommend exploring every option available to make certain that you're only logging what needs to be logged. Examine your Windows audit policy for instance (if you're in a Windows environment). As you probably know, one checkbox there can make a considerable difference in the number of events that get sent to LEM.
The only option I know of that would help you past the 2.2TB limit would be another complete LEM system. Then you would have some systems logging to one, and others logging to another. In your LEM console, you would be able to easily switch between the two LEM 'environments'. As I understand it, nDepth queries can be used to run against either DB, but filters can show traffic from both LEM systems simultaneously. I could be mistaken on those specific details.
Thanks, that is along the lines of what I was expecting. I am curious though, my understanding is that if you need to use one of those backups; when you restore it you will replace your current database with the backup, is that correct?
Prior to 5.6, that was certainly the case. I believe there are other options available with version 5.6 because of the new database, but don't know the details.
When you say 90 days online. Your are referring to being able to query results via ndepth for up to 90 days correct? For someone also in the PCI realm. I would be interested in some of the industry standard best practices that others are leveraging with LEM. We currently do a monthly archiveconfig, and weekly backupconfigs and logbackupconfigs.
It would also be helpful for an actual list of audit policies to enable, this being my first time at the rodeo regarding PCI. For example they say Object Access "Success and Failure". But are they all enabled? I don't really want to turn on audit filtering platform connection..etc and have our logs explode. Any guidance is appreciated. Thanks
Here is what I found and what I have set based on this:
Security System Extension No Auditing
System Integrity Success and Failure
IPsec Driver No Auditing
Other System Events No Auditing
Security State Change Success and Failure
Logon Success and Failure
Logoff Success and Failure
Account Lockout Success and Failure
IPsec Main Mode No Auditing
IPsec Quick Mode No Auditing
IPsec Extended Mode No Auditing
Special Logon Success and Failure
Other Logon/Logoff Events Success and Failure
Network Policy Server No Auditing
File System Success and Failure
Registry Success and Failure
Kernel Object No Auditing
SAM No Auditing
Certification Services No Auditing
Application Generated No Auditing
Handle Manipulation No Auditing
File Share Success and Failure
Filtering Platform Packet Drop No Auditing
Filtering Platform Connection No Auditing
Other Object Access Events No Auditing
Detailed File Share No Auditing
Sensitive Privilege Use Failure
Non Sensitive Privilege Use No Auditing
Other Privilege Use Events No Auditing
Process Termination No Auditing
DPAPI Activity No Auditing
RPC Events No Auditing
Process Creation No Auditing
Audit Policy Change Success and Failure
Authentication Policy Change Success and Failure
Authorization Policy Change Success and Failure
MPSSVC Rule-Level Policy Change Success and Failure
Filtering Platform Policy Change Success and Failure
Other Policy Change Events Success and Failure
User Account Management Success and Failure
Computer Account Management Success and Failure
Security Group Management Success and Failure
Distribution Group Management Success and Failure
Application Group Management Success and Failure
Other Account Management Events Success and Failure
Directory Service Changes No Auditing
Directory Service Replication No Auditing
Detailed Directory Service Replication No Auditing
Directory Service Access Failure
Kerberos Service Ticket Operations Success and Failure
Other Account Logon Events Success and Failure
Kerberos Authentication Service Success and Failure
Credential Validation Success and Failure
Off topic but just wanted to compare with some others. Anyone?
Thanks, this is good info! Funny thing, that site you link to is actually a company that we are considering using for their FIM product.
Yes, for the 90 days, I was referring to an nDepth search. I would also recommend against auditing the "audit filtering platform", "Detailed File Share", or the "Directory Service Access". These all log copiously, and have (arguably) little security value.