Had a network blip last night that caused split brain in our HA (again).....working with TAC at the moment, but I was poking around with SWQL to confirm something and found this...
Beginning to take this personally.
15-20?! Arg you're a machine!
Awesome, thanks @Kita . Don't hesitate to have the engineer reach out to us, as we're all on a call trying to repair these issues without the engineer at the moment.
@danbert The fix in 2024.2.1 is to address the split brain issue. We will dig into why the false positive are occurring
Ha....we'll be building out those scripts.....Solarwinds does not do HA well...
Hi @Kita here is the case #01734665. The engineer fixed the HA, but it went split brain again shortly after. We're aware of the 2024.2.1 fix, but need a stable platform first. Plus we're dealing with approximately 60 Nodes showing as down with false positive alerts that we're waiting for an engineer on potentially related to the first issue.
@danbert If you can share the case number, we can help validate the issue. There are several customer running into this issue in 2024.2 and uncovered a potential fix for these scenarios in 2024.2.1. This was added to the release notes as well.
I always tell my clients that I have had more downtime from fixing split brain scenarios for clients than I've ever seen from actual single server failures. Worst case I can fix a broken single instance pretty quick, often before any users realize anything happened. Fixing a problem and then sorting out all the reg keys and db entries for split brain to get it running again almost always adds another 15-20 minutes to the recovery time. If I had to live with an HA environment full time I'd probably have saved a set of scripts to set all the keys and db entries for HA back to my "default" position and i would fire that off immediately upon any failed HA handoffs.
Cisco Firepower Threat Defense FTD-1.3.6.1.4.1.9.1.2320.ConfigMgmt-CommandsExported with NCM 2023.4.2
AlertSummary.xmlHi All, I've done some work to tidy up our alert definitions (both the Alert Manager and Alert Actions) and the more important alerts have now been tailored and given a "[ *** ]" prefix so it is easy to see the alerts we have catered for versus those that might appear once in a while. I am monitoring this…
I wanted to share a new dashboard a co-worker ( @mblackburn ) came up with to help us better view groups of interfaces and various metrics for each. Using a single HTML widget, we are able to visually compare numerous different interfaces by populating each interface dataset onto its own graph. We are able to view…
PALO-ALG - Logging.xmlPalo Alto Networks ALG Security Technical Implementation Guide :: Version 2, Release: 3 Benchmark Date: 27 Oct 2022