6 Replies Latest reply on Dec 12, 2016 10:53 AM by danielv

    Issue - Rule Creation Logic vs nDepth Logic

    danielv

      I've been having an issue working with nDepth to perform log searches vs. working in the Rule builder. 

       

      When I create a Rule, the logic allows me to pull in logs from various different event sources, including mixing correlation rules with fields from Event Group > Any Alert with fields from Event > TCPTrafficAudit/etc. and there don't seem to be any issues.  Rules trigger as you would expect.

       

      However, when I take my rule logic and try to create a 1-to-1 nDepth search, the logic doesn't function the same, and the query either refuses to build, or returns no results.  I have determined the cause to be mixing event sources.  For instance, you cannot mix fields in "TCPPortScan" with fields from "MailServiceAccess" or "Any Event" in nDepth without getting completely unreliable results.

       

      Is this a known issue?  I would like to be able to test my correlation rule logic by doing a manual search of the rule logic in nDepth over the last day/week/etc. but this seems impossible in the current iteration of the product.

       

      Has anybody else had this issue and found a workaround?

        • Re: Issue - Rule Creation Logic vs nDepth Logic
          jhynds

          Hi Daniel,

           

          Would you mind posting a screenshot of a sample rule you are testing against in nDepth & I'll do some investigating.

           

          Thanks,

          Jamie

            • Re: Issue - Rule Creation Logic vs nDepth Logic
              danielv

              Here's a screenshot comparing a quick rule I threw together to demonstrate mixing fields from different event sources in rule creation, and that it isn't possible to create a 1-to-1 nDepth search since the nDepth search creation tool will not allow you to drag disparate event fields into the same group:

               

              rule vs ndepth.png

               

              The rule on the left generates incidents, so I know that it is finding logs that match the criteria.  However, the nDepth search on the right returns 0 results over the same time frame.

               

              I'm basically looking for a way to test new correlation rules that I am creating against historical log data to see what logs trigger the correlation rule.  I had hoped I would be able to feed the rule logic into the nDepth search to go back over the last day/week/etc. but I am running into this problem.  I have to resort to creating a rule and just waiting to see what incidents will trigger in the future.  Is there a way to accomplish what I'm trying to do?

                • Re: Issue - Rule Creation Logic vs nDepth Logic
                  jhynds

                  When the rule triggers I assume you can see a corresponding 'InternalRuleFilred' event within the Rule Activity filter in the Monitor section?

                   

                   

                  If you then click on the rule in question & then Explore -> Event, this should show you the events that caused that rule to trigger.

                   

                   

                  Using the UserLogonFailure rule above as an example, the rule will trigger if there are 3 UserLogonFailure events within 10 seconds:

                   

                   

                  When I explore the InternalRuleFired event, I can then see which Logon Failure events caused that rule to trigger:

                   

                   

                  Does this work for your rule?

                    • Re: Issue - Rule Creation Logic vs nDepth Logic
                      danielv

                      Yes, this works for mine.  See screenshot below.  But is there a way to bring the rule logic over to the nDepth search so that I can do a historical search of all events that would match that rule logic criteria?  I would like to be able to do this for 2 reasons.  1) Tune my existing correlation rules to eliminate noise and 2) Test new correlation rules without having to implement them then wait for them to fire to see if my results are what I expected to see.

                       

                      RuleFired.PNG

                        • Re: Issue - Rule Creation Logic vs nDepth Logic
                          jhynds

                          Correlation rules & nDepth searches work differently - rules work on multiple events but nDepth queries only work off single shared events.

                           

                          I can see that you have UDPBombDenial & CoreAccess events in order for your rule to fire. You will need to determine what the main trigger is, which is likely the CoreAccess event. You could then note the time frames of that event & then query for other aspects to find events that might have matched during that timeframe.

                            • Re: Issue - Rule Creation Logic vs nDepth Logic
                              danielv

                              Ok thank you, I think that answers the question.  It is unfortunate that we can't perform manual searches against our historical data in LEM for the purpose of testing correlation rules.  Is this a feature that may be added at some point? Where can I submit a feature request?

                               

                              Also, the primary logic in the rule is "Any Alert.ExtraneousInfo = *subtype=ips*" - the rest of the rule consists of negating events that I do not want to see trigger the rule.