How large is your SEM database?

We are in the process of fully deploying Security Event Manager (we've had it since 2015 but it was only collecting from a small subset of nodes).  There will be approximately 100 servers and 1,000 workstations connected.  We are being tasked with retaining 6 years of logs since we are a healthcare organization.  I'm trying to gauge how much storage we are going to need to support this requirement.  So far, we have consumed approximately 30 GB in the previous month, with just the servers added.  Is it common/reasonable for a SEM database to be multiple TB's in size?  Is there something different other organizations are doing?  We have configured the archive, but I'm not sure if this is saving much space.

  • If you are also trying to capture workstations you are probably going to need to segment it across multiple SEM servers.  Have all the servers report to one instance, have some workstations on another, etc etc.  Break it up as much as you have to in order to keep the performance and scalability where you need it.  
    SEM supports a design called multimanager where you link several appliances together to the single UI and since SEM licenses are allocated by the number of nodes it doesnt matter how many servers you deploy as long as you are within the allowed node count for your licenses.

    The tricky part to all this is that very few people actually run SEM in that multimanager mode, so it tends to be rarely mentioned on thwack and isn't particularly well documented.  You'll probably want to talk to support to get it all set up to make sure its done correctly.

  • I think this is the right track please note the log retention is only for systems that contain PHI authentication authorization so separating the PHI data source from non-PHI that do not fall into the 6-year retention might help as well.   For example collection syslog on switches or say access points do not have access to PHI so you might not fall into the six-year retention.  Check with your HIPPA consultant to confirm.