11 Replies Latest reply on Jun 19, 2015 3:12 PM by curtisi

    Collect Raw Logs

    freewill

      Hi,

       

      I have Synology NAS device, where there is no connector for it in LEM.  But this device is capable of sending logs to any Syslog server, configured.

       

      I want to know if I can receive the raw logs in LEM, without being normalized by connectors, as there is none.  And later query those logs using nDepth etc???

       

      Please advise.

        • Re: Collect Raw Logs
          curtisi

          This KB will help you set the LEM up for collecting raw logs, but you'd still need a connector.

           

          http://knowledgebase.solarwinds.com/kb/questions/3295/Configuring+Your+LEM+Appliance+for+Log+Message+Storage+and+nDepth+Search

           

          I also have a Synology NAS in my home network, and I'd love to get it logging to the LEM, but I can't really justify using company resources for a connector for personal use.  If you contact support with a log sample (Presumably the NAS is set to log to the LEM already and we're getting log events in the local facilities, which you can export from the CMC shell under APPLIANCE --> EXPORTSYSLOG) then we can always send that up to dev as a feature request.

            • Re: Collect Raw Logs
              freewill

              Hi,

               

              Thanks for reply. I am aware of the KB and already configured the nDepth.  But as there is no connector/tool for synology I could not configure the last part.  I understand that I can submit sample logs to Dev for connector but what I want to do, at least for the time being, is to collect the log for compliance sake.

               

              I just want logs to be collected and available in nDepth, for now.   I have configured the Synology to send logs to LEM (as syslog) but I am unable to see any logs if I go to cmc>appliance>checklogs .

               

              I have checked each log store.

               

              My understanding is:  connector is for parsing and normalizing but I can get the raw logs and store them in LEM and query via nDepth.

               

              If  my understanding correct then what I am doing wrong here.................

              • Re: Collect Raw Logs
                byrona

                Is it still true that with LEM you are unable to store even raw logs if yo don't have a connector?  If that is the case it seems that this puts LEM at a significant disadvantage when compared to other products on the market.

                  • Re: Collect Raw Logs
                    curtisi

                    byrona: You can do it, and I covered the steps answering this other thread.

                     

                    Re: new syslog node

                     

                    You need a connector, but it can be any connector that can handle the log type (ie, syslog, flat files, evtx) so long as you set the connector output to "nDepth only" it'll skip normalization and store the raw log data.

                      • Re: Collect Raw Logs
                        byrona

                        curtisi: awesome, thanks for pointing that out.  So, is there an easy way to tell what log types the connectors are designed to handle?  I am guessing this is something obvious that I am just not thinking about?

                          • Re: Collect Raw Logs
                            curtisi

                            Uhhh...pick something that logs the same way?  There's not an easy way, I mostly use arcane knowledge and divining (and luck) to get it right if I have no idea what to use.  If it's syslog, I use the Cisco IOS connector.  Everything else is "come as it may."

                              • Re: Collect Raw Logs
                                nicole pauls

                                Be aware when you do this that when you search it in nDepth it will all be "typed" with that connector, so you just need to know what you're looking for. I usually use one that I DON'T already have so I can easily find the messages from it.

                                 

                                There's no secret decoder ring/map of connectors and how/what they are designed to read, though.

                                  • Re: Collect Raw Logs
                                    byrona

                                    So, I assume the use case here is clear and that while this can be done, the solution is less than optimal.  Is there something in the works to hopefully make this type of thing easier and perhaps more user friendly?

                                      • Re: Collect Raw Logs
                                        nicole pauls

                                        We just don't see a ton of people using the raw log support, so it's not something we've focused on. There's nothing in the near term that will change this explicitly (either by documentation, by feature development, or new connectors in the works).

                                         

                                        It's technically relatively easy for us (usually) to create raw log-only connectors provided the file naming and rotation conventions are pretty well known, but as it stands right now without some changes we would be careful about 'releasing' them since it might create confusion as to what we actually support (i.e. normalize) and don't (when I did this for myself internally, I called them "nDepth Only" to make it clear, I couldn't keep them straight myself). Effectively we build a connector that drops all normalized events in case you happen to select that option and has the right category/name.

                                         

                                        A better solution would let you input those things and be more like the "generic" connectors we've discussed on and off (that's kind of what this is anyway). But again... no promises