KiwiSyslog Evaluation - Log4Net and XML

Hi Idan,

I'd start by creating a custom File format in Kiwi Syslog, that just logs the "message" to log file (without all the Date, Time, Priority, etc.).

Setup > Formatting > Custom file formats > Create New custom format, and select just "Message" field (no delimiters, no qualifiers, just the message).

Re-configure your Log to File action (or create a new one):
Action : Log to File - Select the Custom log file format you just created.

eg. "Log4Net.xml" (as might be generated from custom logging format above)

<log4net:event logger="Logger" timestamp="Timestamp" level="Level" thread="Thread" domain="Domain" username="Username"><log4net:message>Message</log4net:message><log4net:properties><log4net:data name="DataName" value="DataValue"/></log4net:properties><log4net:locationInfo class="Class" method="Method" file="File" line="Line"/></log4net:event>
<log4net:event logger="Logger" timestamp="Timestamp" level="Level" thread="Thread" domain="Domain" username="Username"><log4net:message>Message</log4net:message><log4net:properties><log4net:data name="DataName" value="DataValue"/></log4net:properties><log4net:locationInfo class="Class" method="Method" file="File" line="Line"/></log4net:event>
<log4net:event logger="Logger" timestamp="Timestamp" level="Level" thread="Thread" domain="Domain" username="Username"><log4net:message>Message</log4net:message><log4net:properties><log4net:data name="DataName" value="DataValue"/></log4net:properties><log4net:locationInfo class="Class" method="Method" file="File" line="Line"/></log4net:event>
...etc...

You should then be able to set up the output as an external entity in a separate file to form a correct XML file.
eg.

<?xml version="1.0" ?>

<!DOCTYPE log4net:events SYSTEM "log4net-events.dtd" [<!ENTITY data SYSTEM "Log4Net.xml">]>

<log4net:events version="1.2" xmlns:log4net="http://logging.apache.org/log4net/schemas/log4net-events-1.2>
    &data;
</log4net:events>

http://logging.apache.org/log4net/release/sdk/log4net.Layout.XmlLayout.html

We have a KB article also (Setting up XML compliant logging in Kiwi Syslog), which uses a similar (albiet simpler) method, but also includes some XML stylesheet stuff that may be useful...
http://www.kiwisyslog.com/kb/how-to:-setup-xml-compliant-logging-in-kiwi-syslog-server/

Hi Mike,

I followed the first steps you suggested and I have a Log4Net.xml file.

Not sure I understand what to do from there.

Also, Checked the KB you sent and it seems to collide with what you first suggested.

Will appreciate more help on this.

 Thanks,

Idan.

Sorry, I meant to say just to check out the .zip file in the KB article.  (You're right - The article itself will probably not make sense).  I just thought I'd mention it since it touches on the use of XML stylesheets (in combination with XML wrapper)

I haven't actually tried any of this, but I'm assuming that the advice here (http://logging.apache.org/log4net/release/sdk/log4net.Layout.XmlLayout.html) is correct.

It appears similar to how we create the XML_Logfile_wrapper.xml in the KB zip file, but relates specifically to Log4Net's xml logging implementation.

Hi Mike,

Thanks for your reply, but I'm not sure we are on the same page.

My Goal is to have Kiwi Syslog Server parse the "Message" (Log4Net, XML) and show it as columns.

I wasn't able to understand how the links you sent can help me achieve that.

The users of KiwiSyslog will use the Web Interface or the GUI client and they expect to see columns that are based on the received input via UDP, which is the XML formatted text.

Thanks,

Idan.

Unfortunately, you cannot override the default columns in Kiwi syslog (at least, you can't change their names).

You can override the content of the fields though, by using a Custom RunScript action to modify the Syslog Message before it is displayed.

ie.
Rule "Override Syslog Message in display 00"
-Action
+RunScript "OverrideSyslog.vbs" (FULL READ/WRITE permissions)
+Display 00

"OverrideSyslog.vbs"  (Just a sample, based on XML input as specified).

Function Main()

sMessage = Fields.VarCleanMessageText

xmlHeader = "<?xml version=""1.0"" ?>"
Dim oXML
Set oXML = CreateObject("Microsoft.XMLDOM")
oXML.loadXML(xmlHeader & "<log4net:events xmlns:log4net=""http://logging.apache.org/log4net/schemas/log4net-events-1.2"">" & sMessage & "</log4net:events>")

Set log4net_events = oXML.documentElement
For Each log4net_event In log4net_events.childnodes
 Set log4net_event_message = log4net_event.selectSingleNode("log4net:message")
 Set log4net_event_properties = log4net_event.selectSingleNode("log4net:properties")
 Set log4net_event_locationInfo = log4net_event.selectSingleNode("log4net:locationInfo")
 
 message = log4net_event_message.text
 
 logger = log4net_event.selectSingleNode("@logger").text
 timestamp = log4net_event.selectSingleNode("@timestamp").text
 level = log4net_event.selectSingleNode("@level").text
 thread = log4net_event.selectSingleNode("@thread").text
 domain = log4net_event.selectSingleNode("@domain").text
 username = log4net_event.selectSingleNode("@username").text
 
 method = log4net_event_locationInfo.selectSingleNode("@method").text
 file = log4net_event_locationInfo.selectSingleNode("@file").text
 line = log4net_event_locationInfo.selectSingleNode("@line").text
 
 'Syslog Field Override
 Fields.VarPeerName = domain
 
Next

Main="OK"

End Function

More info on overriding Kiwi Syslog's built-in fields, in Kiwi Syslog Help documentation ("the script variables")
You can override all the default fields - Date, time, priority, hostname, message, and populate them with whatever data you want.

Note: The method above of parsing XML data using XMLDOM object is quite expensive, especially when the syslog throughput is high.  Not recommended for high-load situations.

-Kuz

Hi Mike, If I was able to change the formatting to "CSV" would I be able to add columns to the default columns in KiwiSyslog GUI and Web ?

No.  (Sorry).