kstone ✭✭✭✭✭

I have not used an Array load balancer but do have 4 Kiwi syslog servers behind an F5 load balancer. One key point, since syslog is mostly UDP, is to have the timeout for UDP on the balancer rule set to 0 if possible so each packet is handled individually and not as part of a packet stream. For the F5 this is the…

You can use Kiwi and a log forwarder to collect the logs. With the native functionality the best you can do is log the specific entries to a file and use that file for reporting. Anything else will require additional software.

You should configure 'Log File Rotation' to create new files when they get to be over a certain size. You will get multiple files per day. We set ours at 500mb so most editors can read it. If you are searching the file for specific strings you could configure a rule to match those strings and log to another file. This will…

I'm forwarding 1000+ servers to Kiwi syslog without any problems. I'm a little confused by your question, are you forwarding to Kiwi syslog or another Log Forwadrsr?/

The Kiwi syslog engine rated for approximately 2 million messages per hour. Your ingestion rate is a bit over that... The max messages per hour is also impacted by the number of rules, filters, and actions that have been configured. You can do some tweaks that will get you some additional performance but the real solution…

Is there a reason you couldn't filter the messages to exclude the ones you don't want then have Kiwi write directly to the MySQL database? Is the Perl script reading the log files? If so you can create a rule with a filter that, again, excludes the logs you don't want then writes to a file. The Perl script would then read…

I tried a couple different things and don't see a way to make this work without scripting. Maybe someone else has another option.

Log archiving will move the logs from one location to another and, optionally, zip them. We archive our zipped logs daily at midnight to another drive. These are stored then there are clean-up actions that run to delete files older than the retention requirements. The clean up actions run daily, typically around 6am. For…

I'm curious about your architecture that would require it to have 60 syslog servers. That's a lot of capacity. I'll pass this to our SCCM admins. The docs show how to do a silent install from the CLI, it's under Advanced Information-Automating the installation of Kiwi Syslog Server. However, this does not have any…

It is possible. The config file is just an XML text file. We created a 'master' minimal config that then gets copied to each server after the forwarder app is installed. One issue is that if the device does not have the event logs that are configured the Solarwinds forwarder app will not start. For that reason our master…

The change information is not provided in the syslog messages, it only notifies that a change has been made. You can use that message to trigger a script that would gather the Cisco config. Otherwise you need another application(CatTools, Rancid, NPM?) to get that information. If Solarwinds(you didn't say which product)…

This is a very common issue when receiving logs, especially JSON or SNMP traps. Create a script that replaces the double quotes with single quotes. Then it will write the result back to Fields.Rawmessagetext. This would run as an action before your external script action. This could be an action in the same rule or in a…

Can you provide a sample of the message content? Most likely this will require some scripting to set the variable correctly.

If you know the policy names, and there aren't a lot, you can write a rule for each that writes to a log. Or USe one rule and create a script that looks for a string match of the policy names then writes to that filename. Or To be fully dynamic you will have to parse the message(split on the spaces, take the 5th element,…

For #3... have you tried NOT filtering by IP address and accepting any UDP514 traffic? It seems like everything is set up in a way that would work but with NATs you could be getting something unexpected. In the firewall is there a way to see what rules are getting matched? If so, can you determine that the syslog traffic…

Are you looking for a single file that will age out the oldest log lines? If so this is not something that can be done the built in functionality. If you are looking to keep 30 days in a file then create a new file for the next 30 days that is possible. We store the logs per host in daily files and deleted after 90 days.…

I'd like to see: * A native 64 bit version. * Conditional branching in the rules * per server tracking for the timeout filters * a 'not' condition(most likely useful with the conditional branching) * the ability to import MIBs OR a better submisson process * additional methods to send alarms(low disk space, msg count etc)…

What database are you referring to? The web access database? IMO, this is transient data and not an item I would back up. We don't use it so my point of view may not match your needs. One of your rules should be to log everything. This is always my first and default rule. These logs would then get backed up by your normal…

There is no limit on the number of servers, capacity is measured in messages per hour. This is, nominally, 2 million messages per hour. Real world is, in my experience, 1.2-1.5 million max. This is dependent on many factors and we have seen spikes around 2.5 million message per hour that were handled fine. I don't use web…

Check the error log (in C:\Program Files (x86)\syslogd) for any entries relating to the scheduled jobs. If the tasks are running but not successful it could be related to permissions on the files or folders. If the tasks are not running at all it is likely a configuration error(folder doesn't exist, copying to the existing…

The email for less than X messages is useful for other issues not necessarily direct KSS errors. If you normally get 1000 messages in 5 minutes and only got 10 there may be an issue(network, sending hosts, FW, etc). The service and/or process monitor are the most important. Then the log file. Most errors will end up…

What is your requirements to store the logs, especially for 5 years? SQL can certainly do it with the correct configuration but it's hard to make recommendations without knowing the scale required. We average between 5 and 8 billion logs yearly, at the standard message size(1k) it's a fairly large datastore. Since we have…

I've never seen that behavior. Have you checked the startup/autostart options on the host? Is there a login script that could be calling it?

For VbScript you can do a split on the comma into an array then only use the elements you want to create the new message: msgarray = split(Fields.VarCleanMessageText,",") wantedfield1 = msgarray(6) wantedfield2 = msgarray(9) wanteddata1 = split(wantedfield1,"=") wanteddata2 = split(wantedfield2,"=")…

Our usage is very similar with Kiwi as the syslog intermediary for SCOM. The legacy solution is a VBscript that formats the message then calls EventCreate. This allows us to specify the source and event ID but not custom logs. The 'new' solution is a Powershell script that will create the event log if needed and write the…

In your rule create a filter using the Message Text field. Then select the Filter Type "Complex". In the Exclude field you can add the text to be matched and then ignored.

Our team uses SCCM to deploy it. I don't have specifics but I would see it as installing the MSI then copying an existing config file to the install folder and restarting the service. Be aware that if an event log that is included in the config file does not exist on a server that the service will not start. It's best to…

What OS is your Glassfish server running on? If Linux do you have rsyslog available? If you do you can configure it to read the local log file and forward it to a syslog host. If rsyslog is not available Logstash is an option. If Glassfish is running on Windows you can use a forwarder such as NXlog or Logstash to read the…

How are you determining that nothing is reaching the syslog server? If you have not already done so, create a new rule with a filter that includes either the event IDs or message text snippet(or both) that you want to capture. Then add an action to display to a specific display ID( 02 for example). Are you using a Win2008…

A similar question was asked recently: SysLog bkyle​ suggested using a registry key to include more hosts in the daily report. This maxes out at 999 hosts so it doesn't work for us. Our default rule includes an action to log to a file. When configuring this action we use the 'AutoSplit' value to use the hostname as the log…