Collect Raw Logs

This KB will help you set the LEM up for collecting raw logs, but you'd still need a connector.

http://knowledgebase.solarwinds.com/kb/questions/3295/Configuring+Your+LEM+Appliance+for+Log+Message+Storage+and+nDepth+Search

I also have a Synology NAS in my home network, and I'd love to get it logging to the LEM, but I can't really justify using company resources for a connector for personal use.  If you contact support with a log sample (Presumably the NAS is set to log to the LEM already and we're getting log events in the local facilities, which you can export from the CMC shell under APPLIANCE --> EXPORTSYSLOG) then we can always send that up to dev as a feature request.

Hi,

Thanks for reply. I am aware of the KB and already configured the nDepth.  But as there is no connector/tool for synology I could not configure the last part.  I understand that I can submit sample logs to Dev for connector but what I want to do, at least for the time being, is to collect the log for compliance sake.

I just want logs to be collected and available in nDepth, for now.   I have configured the Synology to send logs to LEM (as syslog) but I am unable to see any logs if I go to cmc>appliance>checklogs .

I have checked each log store.

My understanding is:  connector is for parsing and normalizing but I can get the raw logs and store them in LEM and query via nDepth.

If  my understanding correct then what I am doing wrong here.................

Is it still true that with LEM you are unable to store even raw logs if yo don't have a connector?  If that is the case it seems that this puts LEM at a significant disadvantage when compared to other products on the market.

byrona‌: You can do it, and I covered the steps answering this other thread.

Re: new syslog node

You need a connector, but it can be any connector that can handle the log type (ie, syslog, flat files, evtx) so long as you set the connector output to "nDepth only" it'll skip normalization and store the raw log data.

curtisi‌: awesome, thanks for pointing that out.  So, is there an easy way to tell what log types the connectors are designed to handle?  I am guessing this is something obvious that I am just not thinking about?

Uhhh...pick something that logs the same way?  There's not an easy way, I mostly use arcane knowledge and divining (and luck) to get it right if I have no idea what to use.  If it's syslog, I use the Cisco IOS connector.  Everything else is "come as it may."

Be aware when you do this that when you search it in nDepth it will all be "typed" with that connector, so you just need to know what you're looking for. I usually use one that I DON'T already have so I can easily find the messages from it.

There's no secret decoder ring/map of connectors and how/what they are designed to read, though.

So, I assume the use case here is clear and that while this can be done, the solution is less than optimal.  Is there something in the works to hopefully make this type of thing easier and perhaps more user friendly?

We just don't see a ton of people using the raw log support, so it's not something we've focused on. There's nothing in the near term that will change this explicitly (either by documentation, by feature development, or new connectors in the works).

It's technically relatively easy for us (usually) to create raw log-only connectors provided the file naming and rotation conventions are pretty well known, but as it stands right now without some changes we would be careful about 'releasing' them since it might create confusion as to what we actually support (i.e. normalize) and don't (when I did this for myself internally, I called them "nDepth Only" to make it clear, I couldn't keep them straight myself). Effectively we build a connector that drops all normalized events in case you happen to select that option and has the right category/name.

A better solution would let you input those things and be more like the "generic" connectors we've discussed on and off (that's kind of what this is anyway). But again... no promises

A better solution would let you input those things and be more like the "generic" connectors we've discussed on and off (that's kind of what this is anyway). But again... no promises

I agree this would be a better solution.