1 of 1 people found this helpful
First, double check and make sure your "Response Window" (below correlation time) is also wide enough - it should be at least 40 seconds also. The default is 5 minutes, so you should be fine. This accounts for things like clock drift.
Otherwise, everything sounds right. When you get 2 Cisco IOS Logon Failures in 40 seconds it doesn't fire, but when you get a single 1 and change the rule to 1 it does? It might come down to what you have in the correlations box. You'd want something like:
UserLogonFailure.ProviderSID = "%blah-blah-blah"
2 in 40 seconds
You could also use an advanced threshold if you want it to be the same account (DestinationAccount SAME) or source IP (SourceMachine SAME).
You can check the "SolarWinds Alerts" filter in Monitor to see rules firing, or look for InternalRuleFired alerts, which generally let you know what rules are being triggered and when. If the rule is being triggered and the action isn't being fired, it's something with the action, but it sounds like it's that the rule isn't being fired at all.
I let the response window at 5mins and the device and LEM (virtual appliance) are using the same NTP server. - How can I check that LEM is in sync with the NTP server? - I couldn't find a cli command or a menu in the GUI to verify this..
The correlation is very simple, I choose
UserLogonFailure.DetectionIP = 172.20.154.*
2 in 40 secs
no advancde threshold is configured. thats all. Changing this same rule to be
1 in 40 secs
the rule fires...
As I use ssh to logon to the Cisco device I can see a pre-bulit rule for 3 successive failed ssh logons which I can force to fire also. I copied this one and added the SNMP Trap Action at the end - but I do not receive a trap..
When you set the rule to fire on 1 in 40 seconds, does it fire once, or does it fire twice back to back?
If it only fires once, it sounds like only one event is being generated. You could confirm that with a filter in Monitor for the exact same criteria (UserLogonFailure.DetectionIP=172.20.154.*).
that is a good point but with that definition the rule fires each and every time the syslog message comes in ...
so that is not the problem...
and I see an UserLogonFailure Alert for each failed logon
If you use a different action, like "Send Email", do you get one or two?
i tried it with "Send PopUp" and the rule fires successfully; what I observerd is, having "1 in 40 sec" and the successfull "snmp action", I see "null , null " as the value in the "ExtraneaousInfo" field, whereas for the "send popup" there is detailed information about
the receiver IP,
the popup text,
the receiver ID
1 of 1 people found this helpful
Thanks for the info. In the end, I was able to reproduce both things you reported:
- When an SNMP Trap Alert is sent, the InternalCommands message shows "null " where it should show information about the SNMP Trap Alert that's being sent.
- When you use an SNMP Trap Alert as the action in a correlation with a threshold, a trap won't get generated. If it is the ONLY action the rule entirely does not fire. If there is a secondary action, that action fires, but the SNMP Trap Alert does not fire. If you remove the threshold (i.e. set it to 1) the rule does fire and the SNMP Trap Alert is sent.
So, at this point, the workarounds are:
- Fire the SNMP Alert each time the failure happens (set threshold to 1). Kind of a bummer since you get multiples that you don't want.
- Build a rule that infers another alert based on the threshold of 2, then use that alert to trigger ANOTHER rule based on a threshold of 1. Works, but requires an extra hoop for you to jump through. For example:
- Rule 1: UserLogonFailure.DetectionIP=<yourIP>, 2 in 40 seconds, Infer Alert "FailedAuthentication" (make sure to fill out the fields by dragging them from your UserLogonFailure alert into the FailedAuthentication Infer Alert action)
- Rule 2: FailedAuthentication.DetectionIP=<yourIP>, 1 in X seconds, Send SNMP Trap Alert
- If you need additional criteria, you could also use FailedAuthentication.InferenceRule="Rule 1's Name Here" to make sure it's ONLY those events that fire the SNMP Trap.
If you want to see an example of #2, let me know and I'll generate them and export them as examples that you can import.
Can you submit a support case so that we can track and notify you when this issue is resolved? I will file your issue as well since I was able to reproduce consistently. I will also try to follow up in this thread when the issue is resolved.
Edit: Before the community team beats me to mentioning: if you do file a support case, post the # here so we can refer back to it. Thanks!!
Thanks Nicole for reproducing the issue.
In fact I have a support case open, yet; the Support Case # is 367887.
I posted the question here because I wanted to know if I am really the only one who is seeing this issue - and indeed, it seems that I am the only one who is using this feature... ;-).
1) is not an option, because I started with this very simple one, but in the end the logic should be more advanced than just "count it" :-)
2) I yet thought about this possibility and did some quick tests but perhaps I have to do some more advanced because what I saw was that much of the original information gets lost in the SNMP Trap because it fires on the Infer Alert.
Hopefully this gets resolved very quickly, because we bought this software exactly because of this feature.... - I thought it is a standard feature and I didn't test it :-(
- ok, the feature is there - but it is not working ;-)
Now, that you are looking at this I will take the oportunity to point you on my feature request regarding the "Snmp Trap Action" I posted here:
It would be great if there would be (at least) one field where you could use the standard variables (e.g. UserLogonFailure.DetectionIP, UserLogonFailure.DetectionTime, etc.) and add them one behind the ohter (seperated by or added with any free text). Currently, if I drag and drop these vars from the left pain to the fields in the right pane, one var is substituted with the new one and not added (- which is clear for the existing fields!). Having this new field - being the same for all: Alerts (or Infer Alerts) and Actions, I myself can define which Information I want to keep and carry over to new Alerts or Actions.
In the end, its just one new field that I want (or a couple of these :-) ), like Info_1; Info_2; Info_3; ....
For Option #2, if you select the FailedAuthentication alert as your inferred alert, the fields should be nearly identical to the UserLogonFailure alert so you don't lose any field data (until you get to the SNMP step, which relates to your second point ).
Rule #1: 2 UserLogonFailures in 40 seconds, action Infer Alert, use FailedAuthentication (Generic > Security > Suspicious > AuthSuspicious > FailedAuthentication)
Rule #2: FailedAuthentication exists (with criteria), action Send SNMP Trap
Thanks for the feedback on the Send SNMP/Infer actions. Something like what we do in Email Templates where you can concatenate free text with field data would definitely be useful.
I'll make sure your support case gets attached to the bug report.
Currently I am still configuring my rules and correlations and just add an Infer Alert (beside the SNMP Trap Action) when need to see at least that the correlation is working.
And hopefully this bug is resolved very quickly and I just can go on and remove the Infer Alert from my rules and have the configuration working as needed..
I know the workaround is a little cumbersome, but I'm glad it's at least working. We are planning to resolve this issue in an upcoming hotfix (and certainly release). I'll close the loop here when that's available.