I am new to LEM area
There is a LEM that is already setup and running.
Is there a way to tell if we are getting raw logs or ??? from the windows servers, linux and network equipment etc....???
Any help would be great
Thank you
A couple of clarifications to make sure we're on the same page:
Log and Event Manager maintains the raw logs by default for a short period of time, namely in order to "normalize" them which means that the LEM connectors read the data, parse the data, and put key data into specific fields so that it can be searched, correlated, etc.
Typically for most customers maintaining the "raw" events (such as EVTX or Syslog) natively is not required.
If you just want to search for the correlated data to make sure that you're getting events, you can do it from ndepth:
How to use nDepth in SolarWinds Log & Event Manager - Video - SolarWinds Worldwide, LLC. Help and Support
In your situation, if it is required by management or for auditing purposes, the LEM can be configured to retain the raw events in a separate database. If you feel this database has been enabled, this article will discuss confirming that and how to use it:
Search raw log messages in LEM using nDepth search - SolarWinds Worldwide, LLC. Help and Support
If it is enabled, there is a steep resource cost associated with it, so you will want to make sure that it is absolutely necessary in advance and I usually suggest reaching out to Support as well so that they can discuss all of the options available to you. If you would like to know more about enabling the raw database, you can find that article here:
Configure LEM to store original log messages (nDepth log retention) - SolarWinds Worldwide, LLC. Help and Support
Thank you jrouviere, I will look over this information.
From what I was reading about lem requirements, We are using a lot of hardware example 18 CPU's 48GB MA and 48GBMR
We are not running over 500 devices it is really slow, 7days is the max and sometimes crashes.
That is why I asked, about raw logs and a few others things.
Raw logging does have a large impact on required resources so that could definitely be part of it. If you don't need it, I would suggest turning it off.
You will also want to review articles such as this one for performance checks:
LEM performance checks - SolarWinds Worldwide, LLC. Help and Support
I would start with running the Database Maintenance Report and comparing your events per day to this chart from the KB:
Those are two of the first things that I would check, but if there are any Internal errors or messages you're seeing or recent rule configurations that you made, they could play a factor as well. Support can help identify a lot of this very quickly, but that KB article has a lot of the information that you would want to review as well.
Well, I know that we do about 35 to 40 million events per day.
I have not seen any internal errors or messages, I would not know where to look yet.
I have watch all the videos for lem that is on youtube (51 of them). I try watching them off the website, they crash after one min or so no matter the browser or pc lol.
Those videos does not say anything about how to limit logs really.
I still going on over all this.
Thank you very much again jrouviere.
https://support.solarwinds.com/Success_Center/Log_Event_Manager_(LEM)/LEM_Administrator_Guide/0970-Search_raw_log_messages_using_nDepth_search?CMPSource=THW&CMP=DIRECT
I do not have the option for "On the far right of the search bar, move the switch from Events to Log Messages"
To bad there is not a pic of it to verify lol
https://support.solarwinds.com/Success_Center/Log_Event_Manager_(LEM)/LEM_Administrator_Guide/0160-Configure_LEM_to_store_original_log_messages_nDepth_log_retention?CMPSource=THW&CMP=DIRECT#Configure_connectors_to_send_original_log_data_to_LEM
Alert: Sending data to the alert database> they show only alert
https://support.solarwinds.com/Success_Center/Log_Event_Manager_(LEM)/LEM_Administrator_Guide/0160-Configure_LEM_to_store_original_log_messages_nDepth_log_retention?CMPSource=THW&CMP=DIRECT
Rules do not fire on raw (non-normalized) log data. Rules can only fire on normalized data, I see rules firing off
Raw (non-normalized) log messages do not appear in Monitor view in the Console, can not tell on this one
Hey JROUVIERE
Can you give more guidance on the link for LEM performance checks.
I can run reports but I do not know what is off and what is not?
It doesn't sound like you are using nDepth (the RAW database) then. Not to be confused with nDepth, the graphical search tool.
If you are having performance issues than likely it's a reservations or rule configuration issue.
I would run the reports suggested in the performance article and check vs your reservations. Specifically, at a minimum, run the Database Maintenance Report and compare to your reservations using the table I highlighted. If your resources aren't in the same band as your events per day, then increase the reservations. Beyond that, Support will best be able to help you check if there are other configuration issues (over active rules, misconfiguration, etc).
Sorry for the late reply
I start looking at the performance weblink again and try to do a few things I know I can do or I thought I could do.
I have a little trouble find exactly they looking for can you look at this and tell me why I can not find any filter that starts with AnyAlert
nDepth Searches (created from GUI-console, Explore > nDepth)
Rules - InternalRuleFired for the last 24 hours 1714
Agent - InternalUnknownAgent for the last 24 hrs 2079
Agent - InternalDuplicateNode for the last 24 hours (Error: General: Search finished prematurely) Can not find this in the events filter
I do have internalDuplicateconnection 12
Un-matched/Mis-matched data - AnyAlert.EventInfo=*unmatched* for the last 24 hours Can not find this in the events filter, fact can not find any filter that starts with AnyAlert
I can find is unmatched, the event name is listed as internalnewtooldata count 128
When try mis-matched, I get nothing
Incorrect Security Connector - AnyAlert.EventInfo=*vista alert* for the last 24 hours Can not find this in the events filter, fact can not find any filter that starts with AnyAlert
when I use the word vista, over 32 different event names show up results show over 4 million
Windows noise - AnyAlert.EventInfo=*windows filtering platform* for the last 24 hours Can not find this in the events filter, fact can not find any filter that starts with AnyAlert
when I try windows filtering I get 3 different event names and results show 2607
Thank you again
