5 Replies Latest reply on Jan 27, 2016 3:58 PM by whpd

    General Guides I wrote up for performing some basic LEM tasks


      My company just purchased LEM.  While learning how to use use it and getting it set up, I had some initial trouble learning how to perform some tasks that may not be terribly basic, but also are not advanced either.  I found the user guide (http://www.solarwinds.com/documentation/LEM/Docs/LEMUserGuide.pdf) among others, but I didn't find all of them up to snuff for what it was I was trying to accomplish.  Some of this may be duplicate of the guide linked above or available elsewhere, but in either case may be useful for new users of the product, which is the purpose of this post.  I originally wrote this in onenote as a guide for my coworkers, so apologies if the formatting is a bit off.  I'll try to correct it as best as I can.


      Adding sources for logging

      There are two ways to add a source (node) into LEM.  The first is using Syslogs and connectors, typically from network switches or appliances.  To do so, forward Syslogs to the LEM appliance on port 514.  Look up how to do this online if necessary.  You then will need to enable the relevant connector(s) on the appliance.  Go to Manage > Appliances and select the gear icon > connectors.  Search for the relevant connector, then select the gear icon > New.  You can typically leave all fields as default (with a few exceptions) and hit Save.  It will create a new connector below it with a gray icon in the status column.  Select the gear icon > Start and it should eventually turn green.  This indicates that the connector is active. 

      Note: there is a progress/status message and progress bar at the bottom near the bottom right when activity is occuring; on the main window and may be in the grayed out section if there is a box in the forefront.

      You should see a yellow message at the top indicating that a new node has been found, and sometimes that connectors are enabled.  This typically means that you are now receiving syslog events, but you may want to go to Explore > nDepth to find out for sure and what kind.  It is possible there may be no relevant connector.  In that case, you are likely SOL, but maybe google/thwack will be able to save you.


      For Windows/Linux hosts, there exists an installable agent which you can download from the LEM appliance webconsole.  To download it, go to Manage > Nodes and select Add Node near the top of the page.  Select Agent Node and download the relevant installer.  Copy it and run it on the needed host(s), using the Appliance IP when requested by the installer.  Once the installation completes, you should see a yellow message at the top indicating a new node is found and the node should be listed under Manage > Nodes.  Now you need to add relevant connectors.  Do so by selecting the gear icon that matches the node and selecting Connectors.  On the connector configuration screen that appears, find the relevant connector(s) and select the gear icon > New.  Typically this can be left to the defaults, so click Save.  On the new connector that is created, there should be a gray status icon.  Click the gear icon >  Start.  The icon should eventually turn green indicating the connector is active and events should begin coming in.


      Creating/Editing Email Alerts (as of LEM v6.2)

      To actually create or edit the alert, you will need to go to the Build > Rules section.  Typically you want to clone from a template.  Once you are editing your action there are two main parts -- The correlations and the actions.  The correlation portion is the most difficult to set up, especially the first time you create one.  Typically you want to go back to Explore > nDepth, or possibly look at an event in the Monitor section.  You will probably need to come up with a search to try to isolate a relevant event and look at the Event Fields that are important for discerning your desired alert.  Once you figure out a relevant event that you want to correlate to an alert, the first and most relevant part is to look at the Event Name as most correlations start with EventName.SubEventField.  You can typically swap back and forth between the explore tab and build tab without losing your place, so that's a helpful way to go about it.  From the Build > Rule > RuleBuilder screen , you would want to drag an event type from the left menu panel to the correlations section on the right.  Typically you want to first select an Event correlating to the EventName mentioned earlier, then select and drag the relevant sub field from below to the correlations tab.  The specific parent EventName is important, because if you use the wrong one, your rule might never match, and thus be triggered.  On the right side the equals/does not equal and correlation info should be fairly self-explanatory, but keep in mind  you can a wildcards (*) or multiple throughout your statement to generate a precise statement (ex: User * has performed * ).   Again, you will want to reference an actual event for the structure.  Build these with and/or statement chains to narrow down only the specific events you want to trigger your rules.  Reference other rules if you need examples.

      Correlation Time can help to reduce multiple emails/rule triggers when you typically see multiple events for the same type of event.

      To send an email as an action, you can clear any other actions from the rule (or leave them if desired), and select Actions / Send Email Message from the right side and drag to the Actions section.  You will need to use an Email Template, which is explained how to be created/edited below.  The variables from the Email Template need to be linked to either text that will be static/constant (dragged from the constants menu on the left), or from variables derived from the Events/Events Group menu on the left and is used similarly to the way they are used in the Correlation section above.  Another reminder that the parent EventName is also very important here when using their child Event Fields.  The other part to note is the Recipients, which determines who the email alerts will be going to.

      The last thing to do is to Save and then make sure to hit Activate Rules on the base Build > Rules page, otherwise your rule will never go into effect.


      Creating an Email Template

      Go to Build > Groups to begin.  To create a new Email Template, select the + button in the top right and select Email Template, or find a similar one and click the gear to its left and select Clone.  Alternatively you could just edit an existing one, but keep in mind this will affect any existing rules that reference that template.  Most of these fields are pretty self-explanatory.  The main thing to note is that you need to create parameters on the left and use them in the Message field or even the subject.  Each parameter will need to be linked to and Event Field or a Constant in the Rule builder.  An Email Template can be used for multiple rules or can be customized for each rule.



      Hopefully this is helpful to someone starting out with LEM.  Since I already went through the effort of typing it up, I figured I'd also post it here so it might be helpful to anyone.  If you have suggestions, comments, or perhaps other written guides of your own you could post them in the comments.  Perhaps if I ever get to it, or if this is well received, I could type up other sections.

        • Re: General Guides I wrote up for performing some basic LEM tasks

          I wanted to update as I have started to develop a way to monitor for Cryptolocker activity on our file servers.  It's taken a fair bit of testing and is certainly not complete, but I also wanted to pass this knowledge on as a starting point for anyone who might be interested in doing the same on their networks.


          I used the FIM connector on our main file servers and had it set up to monitor all relevant directories and looked for specific file masks so that my entire log wouldn't be filled with various file writes throughout the network.  I used some of the common file behavior as referenced here as well as a few other sources.


          I then created a rule to monitor for some the file changes that I filtered for.  I currently only have mine set to email when activity is detected since it was recently created and I'm still somewhat testing it.  You could also have it lock out the account performing the activities or some other action if you felt confident in that.  One thing to note about the above screenshot, for the files you need to put in *\{filename} like in the DECRYPT_INSTRUCTION examples.  LEM/FIM reports the FileName field as the full path, so without it you'll never get matching results.


          A couple other thoughts I have are to monitor for things like file writes on all files, and then have the rule set to only trigger on a set (abnormally high) number of writes, which would indicate a script or virus activity.  This, as I mentioned earlier, would generate a TON of events, so I opted against it.  If that doesn't bother you then it would be another good option.  Also something I'm considering doing is creating a few "honeypot" type files that would not be edited by any normal user, probably put them alphabetically first in some of the common shares for good measure, and set off an alert on ANY activity these files encounter.  Since in my case these are file shares being written to by other workstations on the domain, I have yet to pinpoint the source workstation doing the editing, but it may not be possible with the the way FIM works/logs.


          Has anyone else set up something similar for Cryptolocker activity?  I'm still working to refine mine so input would be appreciated.  In any case I hope this is helpful for someone else looking to begin monitoring for some common malicious activity.

            • Re: General Guides I wrote up for performing some basic LEM tasks


              Thanks for sharing your work.

              If you look at the Rules section of the eval guide, you will find examples of how you can create correlation rules that tell when there are more than X FileWrites in Y seconds/minutes/hours on the same/different machine+account (or any combination of the FileWrite fields)

              That will help you to be alerted only when there are activity levels you are concerned about.

              • Re: General Guides I wrote up for performing some basic LEM tasks

                A further update... I happened upon some further Cryptolocker (4) info { Cisco Talos Blog: Threat Spotlight: CryptoWall 4 - The Evolution Continues } and decided to add a bit more to my monitoring.  One extra thing I am now looking for is FileAttributeChange.FileName == *\HELP_YOUR_FILES.* and FileCreate.FileName == *\HELP_YOUR_FILES.* Note that if you want to add this to your monitoring and you have it set up as I have documented above you'll need to change the rules and the FIM conditions. 

                I've also had mine going for a while and have not seen any false positives (no instances other than the tests I have done), so I now decided that I want to implement automatic account lockouts if this rule happens to be triggered.  In the Cryptolocker rule I created, I added an Action > Disable Domain User Account.  This presents a couple of problems. The rule detects FileCreate and FileAttributeChange events, so in order to act on either, I'd need to possibly create two actions with the 'Destination Account' matching FileCreate.SourceAccount and FileAttributeChange.SourceAccount.  The main issue, and I'm wondering if anyone can help me figure out, is how to properly define the 'Domain Controller Agent'.  This field does not accept things like Connector Profiles or User Defined Groups.  I have been able to assign it a Constant>Text to attempt to statically assign one of the DCs (which have the Agent installed and the Active Response Connector configured), but this hasn't worked for any tests I've thrown at it.  That leads me to believe I'm not assigning it correctly.  I haven't found any documentation on how to properly assign this, only that it can be done.  Anyone got any ideas?