cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

Log time lengths

Level 9

How long do you keep your logs for? The answer can vary wildly depending on the industry you work for. As an example, most VPN providers specifically note that they do not hold logs, so even if a government requested certain logs, they would not have them. The logs they don’t keep are likely to be only user access logs. They’ll still have internal system logs.

Ultimately keeping logs for long is of little benefit unless there is a security reason to do so. The recent shellshock bug is a great example of when older logs can be useful. You may have a honeypot out in the wild and once a known issue comes to the fore, scan your logs to see if this particular bug has been exploited before it was well known.

Country and industry regulations will also influence the amount of times logs are kept. Many countries require that documentation and logging data be kept for a certain amount of years for any number of reasons.

I’m interested to know how long you keep logs for. What particular logs as well as why that length of time was chosen.

28 Comments
Level 17

30-60 days for most items where I am now... depending on importance there might be a longer duration there.

Most things happen or are discovered soon enough that I do not have to look too far back.

Level 12

30 days at most just depends on what it is.

Level 21

We have different log retentions for different customers.  We will generally follow what ever the customer requests and/or what is dictated by any compliance requirements that the customer needs to comply with.

Level 11

We don't really keep a standard... which is a good point. (Security policies are a must!)

Level 13

I'm not the administrator for our log management system (LEM), but I work with him. I think we keep 6 months at least.

Level 9

This is the answer I get back most of the time. A lot of thought is put into a design, but hardly any on the logging itself. Generally the only logging that gets put into design is what we're logging and where is it logging to.

Level 11

I'm not privy to how long they keep logs but I do know that it seems that the idea has shifted.  It used to be that more information was better, until lawyers jumped in an started suing for those records and then companies started trimming what they kept.  At least, thats the story I was told.   

Jim

Level 11

Good article.  I think log retention is site specific based on local administrative and security requirements.  If you don't tailor log retention to your specific environment, get buy in from those who use or potentially use the data, you will end up failing to meet site requirements when the time comes.

Level 14

We do have a standard that we have to abide by.  We pretty much have to keep all logs for all systems for at least a year.  We do archive anything older than a month, so we only have 35 days of active log lag that we can look at on a normal basis.

Level 9

We often leave them at defaults, which is usually too long or too short... We need to develop a standard across the board.

Level 15

Generally, the clients I work with fall into 1 of 2 groups:

  1. No retention levels have been identified, so they stay at the default for whatever product they are working with.
  2. Security compliance has listed a very particular set of standards for log retention, so we right-size storage and work with that.

Ultimately, aside from compliance requirements, I am not a fan of retaining anything that isn't going to be used at some point. But that may just be my neat-freak side

Level 11

Seems like a common thread in the IT business to have a lack of standards or policies.  I have had that problem as at least two positions I've been at.

Jim

Level 10

I agree with Zack Mutchler on this if your not going to use it, then why waste time and literal space on something that is not needed.

Level 14

Online for real time access, 30 days.  Offline for 365.

Level 9

Our state maintains minimum requirements, not only for State Agency requirements, but state security, too--in accordance with Department of Homeland Security. These numbers are often publicly known, but the intent is to keep these numbers private and 'need to know' for state and national security reasons.

MVP
MVP

retention is usually dependent on what is being logged. Normally 30 days, but some requirements mean much longer.

[edit]

also depends on if the log data is cross-referenced by other sources which will impact its retention

Level 9

I tend to agree. Most logs have a very short 'shelf life' - Anything more than a couple of weeks is generally overkill if you need to tie it up to earlier events. On the other hand, there is nothing worse than having to look at a log that no longer exists. On event arrival that log should be copied elsewhere for safekeeping.

Logs are generally ascii text files and depending on verbosity of the application, they tend to be easily compressed and stored.

Certain logs like honeypot logs are certainly ones to keep for much longer.

Level 10

It seems that it is always driven by the Information Security people (who own the log servers where I work), who are in turn advised by legal.

Level 12

30 to 60 days... depending on the content... but I may archive a bunch of logs if I feel like I might need to check them back later... doesn't happen very often though.

Level 9

30-60 days for online access, offline one year

Level 10

3-7 Days, no more than that. If you don't use them between that time, I think you will not use them on the future, except when you need to comply with policies or Stadards 60 Days

Level 9

Let's say you're keeping logs of an orbiting satellite. That satellite pings aircraft in the sky. Two weeks after an aircraft disappears someone says that maybe your logs contained data about the position of that aircraft. By this time your logs are already deleted.

Surely 3 - 7 days is not enough? 99% of the time you won't need anything after that, but there is nothing worse than needing a log a couple of weeks after an event, only to see those logs gone.

Level 16

This is one reason we purchased the LEM.  We need to keep logs for about 90 days, although it depends on the system.

  Firewalls = 100 days

  Router ACL logs = 90 days

  Switch logs = 30 days

  Websense logs = 100 days

  Application logs = 20 to 90 days

  etc.

Issue is that the LEM doesn't let you retain based on category.  Hopefully soon.

We don't even deal with compliance, this is all for troubleshooting and trending.

Level 13

As long as possible. Logs are a gold mine of historical security, performance, and operational data. Some recent examples:

1) Approximating long-term reliability of several WAN links. Routing protocol state changes are a decent proxy for heavy packet loss events, so by mining our long term syslog history for them I was able to get a sense of circuit reliability over time.

2) DHCP logs to identify misbehaving devices. Enriching DHCP logs with MAC OUI data helps a lot with this.

3) Auditing the change history of sensitive AD groups over a long period of time.

4) Identifying the frequency of firewall configuration changes to determine if additional people need to be trained on firewall management.

5) Identifying the chain of events leading up to malware infections.

It's nice to have recent logs available in an easily accessible, searchable format, and archived logs available as compressed plain text. Syslog data typically compresses at least 10:1 if not much higher using gzip, so storage is less of an issue. I like gzip format because you can start a search job using the zcat or zgrep commands from Linux or cygwin and just leave it running until it finishes.

Level 9

30 to 90 days, it depends.  Some log we need to keep for a long period so we can track changes.

MVP
MVP

As a general rule logs are kept for at least 30 days. But it does depend on what it is and how much space there is to store the logs.

Level 11

We keep our logs as long as possible. When we think we have overkilled the keeping of them only then do we purge the logs. You just never know when you may need to look back at something.

Level 15

As a general rule we keep logs for around 90 days.  It depends on space and device performing the logging.  Thanks!