One of the things that I get asked about a lot is whether it's best to use centralized vs. distributed syslog servers within your overall network management strategy. It's a great question as there are advantages to each, so let's take a look...
Centralizing Syslog Collection
There are several advantages to centralizing the collection and storage of your syslog messages. An obvious advantage is that you'll have only one place to look when searching for log messages. This can be a huge advantage if you search often or if you have a large volume of log messages. Another advantage is that in a centralized collection system you only have one system to maintain (assuming that you can scale a single system to meet your needs). This will simplify your management of upgrades, alerts, and rules.
Distributed Syslog Servers
The downsides of using a centralized syslog collection architecture are the upsides of using a distribued design. For instance, in a centralized system the log messages are shipped across the network to the centralized collector. Depending upon your network topology, available bandwidth, and the volume of syslog messages this could be a very bad idea as the log messages may be competing with other applications for bandwidth. Additionally, because most log messages are transported as low-priority UDP messages the reliability of your log storage system will be better the closer the syslog server is to the device generating the log messages.
Scale is another readon to consdier distributing your syslog servers. Many network devices create a lot of log messages. Some networks generate several thousand log messages per second. In these environments, consolidating all of these messages into a centralized database may not be cost effective or even possible.
Combining the Methodologies
Many people today use a combination of the above methods to accomplish their goals. For instance, many of our cusotmers here at SolarWinds buy and deploy the Kiwi Syslog Server as a distribued syslog server - putting copies in each data center or anywhere that they have a collection of devices generating log messages. The Kiwi Syslog server stores the messages locally and provides an interface for local admins to search/view any messages that they need. Then, using the rules/alert engine within the Kiwi Syslog Server they foward any enterprise management/network management related messages up to Orion.
Orion then fills the role of the the centralized NMS and syslog server and has the ability to do additional alerting and to tie the syslog messages back to the original devices since Orion's monitoring those devices and the Kiwi Syslog Server has the ability to forward syslog messages using the original source address.
This system works well when managing network devices such as routers, switches, and firewalls and also works great when managing servers. Historically syslog was only available on Unix/Linux based system but we've also just released a new tool called the "Log Forwarder for Windows" which can be pushed out to your Windows servers and converts standard Windows event messages to syslog messages and then forwards them along to your syslog servers. The Log Forwarder for Windows is free to SolarWinds customers with active software maintenance.
Ping me back if you have any questions about syslog server setup or configuration and as always, flame on...