This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.
You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

Single instance of an Alert

How can I get an alert to only trigger 1 time whether it finds 1 or 1000 node sensors to trigger on?

Then wait X minutes before allowing itself to be triggered again, and only if the conditions are still met (which they won't be because the trigger action is going to take care of all the conditions at once).

I don't care about acknowledgement as it's going to be an informational alert.

Parents
  • I don't believe that you will be able to get only one alert if the Trigger is based on Harware Sensor since each sensor is a unique entity in the database. You could get around this by creating a Group of the sensors on each node than alerting off Group Member Status and including a list of the sensors that were in alarm in the Trigger action.

  • Grouping would have also been my next idea. Do you need any Input Variables for the script to run?

    Also if you are disabling the hardware sensors when they fail... why not diable all on a regular basis as you probably don't care about the status.

  • I'm not passing anything from the Alert to the script. 

    Think of the Alert like a watch dog, if ANY of our interface sensors are found to be enabled matching the names listed in the condition, run the script to find and disable them. The reason I'm not passing the data to the script is that each time we add a Switch, thousands of sensors for interfaces come in and we don't want thousands of alerts triggering thousands of powershell scripts. Some of those interfaces are administratively shutdown, yet, the sensors remain active and turn the device RED because it thinks it's a real issue on an active interface sensor.

    Unless some way exists to NOT monitor interface sensors but keep monitoring for all other HWH sensors??

    Anyway, I'm wondering, what if I added criteria to say "if this alert is currently active don't run"?

  • , given the way that SolarWinds evaluates alerts having it alert on a hardware sensor would have to be a really complex query in order to scope it down. Which would mean probably more prone to failure. It makes me wonder if you'd be better off just making that script a scheduled task that just executes every so often, or making the alert trigger on the addition of a switch device just to avoid the thousands of alerts caused by looking at hardware sensors specifically. 

  • I would agree, it may be easier to just have your script pull down the information from the database and make the changes outside of using the alerting engine altogether. Then set up a scheduled task to run the script every few hours. If you woudn't mind sharing the script you are using I would like to do the same in our environment. I am constantly going out and disabling these as well.

  • $dbserver = "server.domain.com"
    $database = "SolarwindsOrion"
    
    $queryWhere = "where (displayname like '%transceiver%' or displayname like '%bias%' or displayname like '%receive%'or displayname like '%transmit%' or displayname like '%supply voltage%') and isdisabled = '0'"
    
    $cn = new-object System.Data.SqlClient.SqlConnection("Data Source=$dbserver;Integrated Security=True;Initial Catalog=$database;");
    $cn.Open()
    
    $SqlCmd = New-Object System.Data.SqlClient.SqlCommand("SELECT * FROM HWH_HardwareItem $queryWhere" , $cn);
    $SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
    $SqlAdapter.SelectCommand = $SqlCmd
    
    $DataSets = New-Object System.Data.DataSet
    $SqlAdapter.Fill($DataSets)
    
    $command = New-Object System.Data.SqlClient.SqlCommand("Update HWH_HardwareItem set IsDisabled = 1 $queryWhere", $cn);
    $rows = $command.ExecuteNonQuery();
    
    $cn.Close()

  • The issue in basing it on new switches would be when we change the scope of what's to be disabled, as well as discovering new active interfaces potentially turned "on" after being administratively down. So scheduled task makes the most sense I suppose if an Alert can't be utilized. Since we only want to target specific hardware sensors we have to get granular and it's most easily accomplished with a SQL query updating the right table.

    I was even thinking of setting a custom property on our MPE for this 1 alert and have the first alert simply set the custom property to ACTIVE when the criteria is met and include in the criteria to only run if the CP is NOT ACTIVE, then a 2nd alert that only watches for the CP to be ACTIVE and run the SQL/PWSH script, then have it reset the CP to NOT ACTIVE when complete. 

    This should ensure that the 1st/2nd Alerts can only run 1 time, however as it turns out you cannot use a CP with an Alert Trigger Condition when it's pulling from "Other Objects" it seems, specifically, in this case, pulling from Hardware Sensor(Node)! Then I thought, ok I can circumvent that with yet another PWSH that sets the CP accordingly rather than being done by SW itself but it all feels so overcomplicated for what feels like something that could be accomplished with a "don't re-trigger for x minutes if already triggered" option. 



Reply
  • The issue in basing it on new switches would be when we change the scope of what's to be disabled, as well as discovering new active interfaces potentially turned "on" after being administratively down. So scheduled task makes the most sense I suppose if an Alert can't be utilized. Since we only want to target specific hardware sensors we have to get granular and it's most easily accomplished with a SQL query updating the right table.

    I was even thinking of setting a custom property on our MPE for this 1 alert and have the first alert simply set the custom property to ACTIVE when the criteria is met and include in the criteria to only run if the CP is NOT ACTIVE, then a 2nd alert that only watches for the CP to be ACTIVE and run the SQL/PWSH script, then have it reset the CP to NOT ACTIVE when complete. 

    This should ensure that the 1st/2nd Alerts can only run 1 time, however as it turns out you cannot use a CP with an Alert Trigger Condition when it's pulling from "Other Objects" it seems, specifically, in this case, pulling from Hardware Sensor(Node)! Then I thought, ok I can circumvent that with yet another PWSH that sets the CP accordingly rather than being done by SW itself but it all feels so overcomplicated for what feels like something that could be accomplished with a "don't re-trigger for x minutes if already triggered" option. 



Children
No Data