8 Replies Latest reply on Apr 27, 2012 1:24 PM by dcataldo

    LEM with Kiwi syslog Daemon Service


      So I am testing Log & Event Manger and I am trying to pull in the syslog data.  I was informed we were using Kiwi syslog server so after doing research I just deployed the agent to my syslog server thinking it would pull the data.  Unfortunately nothing just windows logs from that server. I log into the server and saw we are using Kiwi syslog Daemon v8.1.3.


      My questions is it still possible for LEM to pull in the data or do I have to setup Kiwi to duplicate the syslog data and transfer over to LEM.  I really don't want to duplicate the data over the network.


      Or if I update to a new version of Kiwi will this resolve the issue? Keep in mind this is the free version of kiwi and most likely my boss does not want to pay for the full version.



      Thank You.

        • Re: LEM with Kiwi syslog Daemon Service
          nicole pauls

          I don't think the Kiwi version matters and we should be able to pull in the data. You are most likely just missing a (non-obvious) configuration step. We pre-configured the Windows integrations, but we're not doing any discovery of the syslog data to pre-configure anything, you'll need to do that manually.


          The second step after you've deployed the agent is to configure the right connectors to monitor your Kiwi logs. You'll want to go to Manage > Nodes, identify the node where your Kiwi syslog server lives, then go to Gear > Tools. From here, you pick the types of devices that are syslogging to kiwi, specify their log path, and that'll pull in the log data.

            • Re: LEM with Kiwi syslog Daemon Service

              Thank for the quick reply,  In am new to this software so I am not quite understanding what you mean by I have to manually configure the discovery in the syslog data.  Do you mean Kiwi server or the LEM Appliance?  If on the LEM appliance how would I go about doing that? 


              The next step I added the devices to the syslog node.   I selected the device; So for example i selected Cisco PIX and IOS and then edit the path of log were files are saved (On my syslog Server) and then hit start but still nothing. 

                • Re: LEM with Kiwi syslog Daemon Service
                  nicole pauls

                  As long as your data is already being syslogged to your Kiwi server, you just need to configure it on the LEM appliance.


                  If it's Cisco PIX/ASA/FWSM/IOS data...

                  1. Go to Manage > Nodes
                  2. Click on your Kiwi Syslog Server Node
                  3. Click the gear to the left of the node and click "Tools"
                  4. Click on the Cisco PIX and IOS tool (you can use the box on the top left to search - typing "cisco" narrows it down a lot!)
                  5. Click the gear to the left, and click "New"
                    1. In the "Alias" box, you can leave it as default, or change to something like "Cisco Firewalls", whatever works - it's just a label.
                    2. In the "Log File" box, type in the path to the Kiwi log file that contains the Cisco data on disk. For example, if it's in the catchall log and that log is in the "logs" directory, use: C:\Logs\SysLogCatchAll.txt
                    3. Click "Save" at the top
                  6. Click the gear to the left of your new item ("Cisco Firewalls") and click "Start"


                  At that point, you should see data coming in to other parts of your Console, check the Monitor view and the "All Alerts" filter to see everything.


                  The biggest detail is knowing what the name of your Kiwi log file is and where it's located on disk. Once you've put that in the "Log File" path box, you should be good to go.

                    • Re: LEM with Kiwi syslog Daemon Service

                      Ok I was doing everything right expect, I forgot to add the actually text file ( Face Palm)  I was just putting the path without SysLogCatchAll.txt  lol...

                      Thank you for quick response and for your help

                      now I can let testing begin =)

                        • Re: LEM with Kiwi syslog Daemon Service
                          nicole pauls

                          Awesome! Glad to hear it was something easy.

                            • Re: LEM with Kiwi syslog Daemon Service

                              One more question,  since LEM is reading SysLogCatchAll.txt ( which is real time)  is there a way to read the archive data.  So for example after today kiwi is going to archive SysLogCatchAll.txt >> 2012-04-27-SyslogCatchAll.txt.


                              What I was thinking of doing is make another device under syslog server but point path directory  to all were archive logs and have *.txt are and make costume filter so I wont get confused with real time data.


                              Or is there a easier way?


                              Thanks Again

                                • Re: LEM with Kiwi syslog Daemon Service
                                  nicole pauls

                                  Regarding real-time data, after LEM reads in the logs it stores a copy of the data in its own database for future reporting & searching. To search that data, it doesn't have to reach back out to Kiwi, it just uses the local copy. Once the data is read in, it's fine for Kiwi to rotate the logs, because we've already copied the data.


                                  As for reading historical data, that's much trickier. LEM was primarily designed for real-time data feeds - set it up and monitor from now forward. If you created a connector and pointed it to the old logs, it would seek to the end and wait for new data, which would never occur, so that won't work.


                                  If you really want to read in that older data, the honest to god easiest way to do it is:

                                  1. Create a new dummy text file (OldSyslogData.txt)
                                  2. Inside the LEM console, create a new copy of the LEM connector/tool to watch that OldSyslogData.txt log (you'll also have to name it something different)
                                  3. Copy the data from whatever historical SysLogCatchAll.txt file you want to read in (e.g. 2012-04-27-SyslogCatchAll.txt) into the OldSyslogData.txt file
                                  4. Save it
                                    1. LEM will read in the data, since it appears as "new"
                                  5. Repeat with whatever days of data you want to feed in.


                                  There's a few other ways that involve creating a connector and rewinding it to read from the beginning but since we're talking about Kiwi syslog data this is the most straightforward.