I needed to create an alert every time that the HA state on any Palo Alto firewall changes. There are some other examples on Thwack but they require you to update your alert conditions every time you add another PA firewall. I needed something dynamic.
This is what I have setup and it has been working great! Wanted to share!
Requirements:
1. You must setup a custom SNMP OID monitor (Universal Device Poller/UnDP). I'm monitoring OID: 1.3.6.1.4.1.25461.2.1.2.1.11 (panSysHAState) which returns the text value of the HA state of active, passive, or disabled.
2. I need to have an Orion Node Custom Property that stores the value of this HA state too. The alert which I'll dive into later has a SWQL query which is looking for the specific Custom Property name of PaloAltoHAState. You'll want to create a custom property value identical to this, otherwise you'll have to update the SWQL query and alert action.
Just a single text format is all we need.
Putting the alert together
*Note: If you did not follow the 2 required steps above then this alert will not work.
**Note: If you want an 'easy button' on building this out I exported my alert and attached it to this document. So you could easily import it into your Orion deployment! So everything after this point is just explaining how it all works. Yay!
Ok, so building the alert. It is a Custom SWQL Node alert. I'm just joining the UnDP table with the Orion Nodes tables here. I'm looking for the OID we built earlier and the alert triggers when the OID value (panSysHAState) does not match the PaloAltoHAState Custom Property value.
Here is the SWQL text just encase anyone wanted to further customize it.
JOIN Orion.NPM.CustomPollerAssignmentOnNode cpa ON Nodes.NodeID = cpa.NodeID WHERE cpa.CustomPollerOid = '1.3.6.1.4.1.25461.2.1.2.1.11' --OID to monitor Palo Alto HA State AND Nodes.CustomProperties.PaloAltoHAState != cpa.CurrentValue --If Custom Property value does not equal UnDP value AND Nodes.Status != '2' --Node not in a down state
On the Alert Trigger Conditions I have two escalation levels.
Escalation 1:
- Send Email alert
- Send info to NetPerfMon log
Escalation 2: Wait 10 minutes
- Update PaloAltoHAState Custom Property value to match the OID panSysHAState value. This effectively clears the alert.
I'll see the alert when it first triggers. This would technically trigger two alerts.
- For the PA firewall that moved from 'active' to 'passive'
- For the PA firewall that moved from 'passive' to 'active'
I get an email notification.
Once escalation level 2 kicks it, it resets everything and the alert automatically clears. Feel free to increase the time between escalation levels from 10 minutes to something higher if you wanted the alert to remain active longer.
Hope you enjoyed this and I hope you found it hepful!
If you want to use the full SWQL query that I used to build out this scenario, here it is.
SELECT Nodes.Uri, Nodes.DisplayName , Nodes.CustomProperties.PaloAltoHAState , cpa.CurrentValue FROM Orion.Nodes AS Nodes JOIN Orion.NPM.CustomPollerAssignmentOnNode cpa ON Nodes.NodeID = cpa.NodeID WHERE cpa.CustomPollerOid = '1.3.6.1.4.1.25461.2.1.2.1.11' --OID to monitor Palo Alto HA State AND Nodes.CustomProperties.PaloAltoHAState != cpa.CurrentValue --If Custom Property value does not equal UnDP value AND Nodes.Status != '2' --Node not in a down state
Changelog:
2021-AUG-27: Fixed a bug in the alert trigger action when updating the custom property. It would not set the correct value when several PA devices were present.
