How many messages can Kiwi Syslog manage?
The licensed version of the software can handle around 2 million messages per hour and the Free version about 300000 per hour. The licensed version has been regularly tested to handle 400-600 messages per second while logging to file.
The licensed version has been increased to have a 20000 message buffer, while the freeware has a 500 message buffer.
If you suspect that you may be losing messages then have a look at the File > Debug options > View message buffer option to check that the "Message Queue overflow:" value is always 0. This indicates the amount of messages that have been dropped. If you are running the Service version then this same information can be found from the Manage > Debug options menu.
To decrease the amount of messages being displayed, you may want to modify your device configurations to only send messages that meet a set level.
If the volume of syslog messages you send to Kiwi Syslog exceeds the above recommendations, you may experience instability and you should consider distributing the load to another installation of Kiwi Syslog Server.
Load Balance Kiwi Syslog Server
Overloading in Kiwi Syslog Server manifests in a couple of ways.
The first (and most obvious) way, is when there is a non-zero value in the "Message Queue overflow" section of the Kiwi Syslog Server diagnostic information. A non-zero value indicates that messages are being lost (due to overloading the internal message buffers). To view diagnostic information in Kiwi Syslog Server, go to the View Menu > Debug options > Get diagnostic information (File Menu > Debug options, if running the non-service version).
The second way, is a little harder to discern, but is most obvious when the "Messages per hour - Average" value in the Kiwi Syslog Server diagnostic information is above the recommended "maximum" syslog message throughput that Kiwi Syslog Server can nominally handle. This value is around 1 - 2 million messages per hour (average), depending on the number and complexity of rules configured in Kiwi Syslog Server.
If either of these two scenarios is true for your current Kiwi Syslog Server instance, then load balancing your syslog message load can mitigate any overloading that may occur.
To load balance Kiwi Syslog Server, start inspecting your Kiwi Syslog Server diagnostic information, specifically looking for syslog hosts that account for around 50% of all syslog traffic. These higher utilization devices are candidates load balancing, through a second instance of Kiwi Syslog Server.
For example, consider the following "Breakdown of Syslog messages by sending host" from the diagnostics information.
Breakdown of Syslog messages by sending host | ||
Top 20 Hosts | Messages | Percentage |
162.19.168.153 | 143054 | 23,92% |
162.19.168.136 | 121773 | 20,36% |
162.19.168.154 | 30102 | 5,03% |
162.19.169.100 | 29908 | 5,00% |
162.19.169.83 | 28576 | 4,78% |
162.19.168.86 | 26452 | 4,42% |
162.19.168.21 | 17897 | 2,99% |
162.19.169.4 | 12809 | 2,14% |
162.19.169.36 | 6780 | 1,13% |
| ... | ... | ... |
From these diagnostics, you can see that 162.19.168.153 and 162.19.168.136 account for ~50% of the syslog load. We normally just start adding utilization figures from the top of the list, until we get to about 50%. Most of the time 50% of all syslog events come from one or two devices, and this is indeed the case here.
To enable a load balanced Kiwi Syslog Server configuration, perform the following actions:
- Install a second instance of Kiwi Syslog Server (on a second machine).
- Replicate the config from first machine to the second.
On the original instance – (File Menu > Export Setting to INI file).
and on the new instance – (File Menu > Import settings from INI file). - Reconfigure devices 162.19.168.153 and 162.19.168.136 to send syslog events to the new instance.
For more information about Kiwi Syslog, see this link Syslog Server and CatTools Network Configuration Manager | Kiwi
Download the Free version here: Free Syslog Server | Kiwi Free Edition