5 Replies Latest reply on Feb 10, 2016 3:58 PM by nicole pauls

    Roadblock Creating Multi-Event Correlation

    danielv

      I'm having an issue creating a multi-event correlation due to the way that LEM parses certain logs, in this case - logs from a NGFW.  All of my IPS logs get parsed and placed into a variety of different event types depending on what they are.  This results in about 20 or so different event names that my logs fall into. (This wouldn't be an issue if all my IPS logs fell into an "IPS Event" event name and the signature fell into the event info bucket - but I digress.)

       

      This is problematic when I want to create a correlation along the lines of:

      10 or more IPS events from the same source IP within XX minutes.

       

      I know that in the monitor section I can create a filter to group up all these different event names and place them back into an "IPS Events" container.  Is this possible to do for correlations? If not, it's very difficult to create an efficient correlation where the source IP needs to be the same across 20 different event names, if that's even possible with LEM in it's current iteration.

       

      Anybody having a similar issue?  Thought of any work-arounds?

        • Re: Roadblock Creating Multi-Event Correlation
          HolyGuacamole

          Yes, you can create Event Groups from the Build > Groups section and use the event group in the correlation rule

            • Re: Roadblock Creating Multi-Event Correlation
              danielv

              Just tried this.  Custom event groups can be used in correlations... however, when going into the advanced thresholds settings, you cannot select Source IP from within your custom event group to restrict events with the "same" modifier.  So, using the custom event groups only gets me halfway there.

                • Re: Roadblock Creating Multi-Event Correlation
                  HolyGuacamole

                  When yoi use an event group, only the fields that are common to all of them are available for those advanced settings. Are you sure source ip is available for all the events in your event group?

                    • Re: Roadblock Creating Multi-Event Correlation
                      danielv

                      This seems to be the issue.  I narrowed my event group down to 3 event types, and I was able to perform advanced settings on source IP.

                       

                      While that is the answer... I can't help but be frustrated by the lack of functionality here.  How would I even know which event types have Source IP as a valid parsed field unless I pore over documentation for each event type while I'm building my custom group?  I can sense the headache already.

                       

                      Wouldn't it be easier if LEM just categorized logs more efficiently? Like, putting all my IPS Events into a single bucket for me to do what I want with, rather than sending them to dozens of event types that are unknown to me? For example, here's my proposed change.

                       

                      How it is now:

                      Event Type>

                                UDPPortScan

                                UnusualICMPTraffic

                                HTTPInvalidFormatAccess

                                CoreAccess

                                UDPBombDenial

                                TCPPortScan

                       

                      How it SHOULD be:

                      Event Type>

                                IPS Event

                                          IPS Event Subtype>

                                               UDPPortScan

                                               UnusualICMPTraffic

                                               HTTPInvalidFormatAccess

                                               CoreAccess

                                               UDPBombDenial

                                               TCPPortScan

                       

                      We're missing a critical level of categorization here.  What events does LEM currently consider to be IPS events? What events parse the Source IP? Who knows!? How can I account for these unknowns while trying to build accurate and complete correlations?

                        • Re: Roadblock Creating Multi-Event Correlation
                          nicole pauls

                          The event-centric not device-centric approach to the event taxonomy can certainly be confusing, and frustrating to reverse engineer. The connectors decide where to map something when they parse the event. The reason they aren't device-centric is that multiple types of devices actually generate those events - firewalls, IPSes, operating systems, basically everything generates a Logon for example. This makes it easier to create rules that say something like "when someone logs on using admin to anything, that's bad" versus having to say "when someone logs on to windows, or my IDS, or my firewall, or their VPN, or, or,....".

                           

                          That said, there IS an event-centric hierarchy in the events - if you toggle from the alphabetical to the tree view, you see something somewhat like what you have there - where the events go:

                           

                          Security Event >

                          Recon >

                          Scan >

                          Port Scan

                           

                          vs.

                           

                          Security Event >

                          Attack >

                          Network Attack >

                          Core Access

                           

                          (this is paraphrased, I don't have it in front of me!)

                           

                          The fields are inherited at different points in the tree - you might acquire fields from Scan, from NetworkAttack, and at the top level with the base dozen or so fields. You can accrue them all the wya to the end, for example, VirusAttack might be one of few events with the VirusName field.

                           

                          Your frustrations are valid, it's a problem the LEM team has been trying to find a way to solve really well for quite a while. Hopefully they come up with something