Is your auditor getting under your skin? What if you told her that she could have all the logs from ALL your routers, ALL your servers and many other devices for the past few years - would that keep her out of your hair for a while?
Fortunately, SolarWinds offers a product in Kiwi Syslog Server that allows you to hold on to logs as long as you want, and not a day longer, with individual devices or networks logging to their own set of managed log files.
Log Retention Best Practices
1) Plan to Keep All Your Logs for Several Years
Every industry is regulated differently, and businesses are often subject to different tax, liability and privacy regulations in different locations. Some common recommended retention periods include:
- Internal Revenue Service (US IRS): seven (7) years
- Payment Card (PCI DSS): one (1) year
- California Franchise Tax Board (CA FTB): four (4) years
- DISA Security Technical Implementation Guides (STIG): one (1) year
- Many State Revenue Departments: three (3) years
- HIPAA Section 164: six (6) years
In most cases it is wise to plan to retain your logs for several years, with "seven years" serving as a safe common denominator.
2) Draft and Approve a Retention Policy
A written, mandatory policy for document retention and destruction is standard operating procedure for publicly traded companies operating under Sarbanes-Oxley (SOX), but it is also a good idea for other companies as well. A written policy, approved by legal council and senior management, gives the IT department the requirements and authority to shape document retention, including logs.
There are many sample retention policies available online, such as this document retention policy template provided by the University of Wisconsin.
3) Automate Log Archival and Retention
To avoid manual mistakes and interruptions, you should automate every possible aspect of your log archival and retention process.
- Collection: use Syslog or SNMP traps to collect logs from every possible source
- Archival: set up your "to disk" logging rules to log separate logs for each device and write a new log for each device each day
- Retention: set up file compression rules to reduce the space used by logs after a few days, then use file deletion rules to automatically delete logs more than a certain number of years old
How to Automate Log Retention with Kiwi Syslog Server
- Download and install Kiwi Syslog Server.
- Configure your routers, computers, applications and other sources to log to the syslog server.
- Split each source into its own file. For each source:
- Create a new Kiwi Syslog Server rule.
- Add a "IP address" filter to the rule that matches the source's IP address.
- Add a "Log to File" action to the rule to log to a specific file.
- Use a file name that contains "%DateISO", such as "router_192-168-1-1_%DateISO.log", to get a different file for each day
- Create a Kiwi Syslog Server Schedule that runs every day and moves old files into a compressed archive.
- Create a new "Archive" schedule and set the frequency to "Day."
- Point the source to your log folder. Keep a file mask of "*.*" to select all log files.
- Set a file age of about seven days. (Only keep what you need for current analysis.)
- Point the destination to a separate log archive folder.
- Make sure the "Move files...", not the "Copy files..." option is selected.
- On the "Archive Options" tab, check the "Zip files after..." option.
- You many also want to increase the compression level.
- On the "Archive Notifications" tab, you may want to set up an archive report.
- Create a second Kiwi Syslog Server Schedule that runs every day and cleans out the archive folder.
- Create a new "Clean-Up" schedule and set the frequency to "Day."
- Point the source to your log ARCHIVE folder. Keep a file mask of "*.*" to select all log files.
- Set a file age of about seven years. (Keep as many years as your retention policy requires.)
- On the "Clean-up Notification" tab, you may want to set up an archive report.
To try this procedure in your own deployment, download a free, full-featured trial today.