cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

NPM 12.2 is Now Generally Available!

Level 15

NPM 12.2 was made available in the Customer Portal on September 13th!  The release notes​ are a great place to get a broad overview of everything in the release.  Here, I'd like to go into greater depth on  Network Insight for ASA including why we built it and how it works.  Knowing that should help you get the most out of the new tech!

Network Insight

We live in amazing times.  Every day new technologies are invented that change how we interact, how we build things, how we learn, how we live.  Many (most?) of these technologies are only possible because of the relatively new ability for endpoints to talk to each other over a network.  Networking is a key enabling technology today like electricity was in the 1800s and 1900s, paving the way for whole wave of new technologies to be built.  The better we build the networks, the more we enabling this technological evolution.  That's why I believe in building great networks.

A great network does exactly one thing well: connects endpoints.  The definition of "well" has evolved through the years, but essentially it means enabling two endpoints to talk in a way that is high performance, reliable, and secure.  Turns out this is not an easy thing to do, particularly at scale.  When I first started maintaining, and later building networks, I discovered that monitoring was one the most effective tools I could use to build better networks.  Monitoring tells you how the network is performing so you can improve it.  Monitoring tells you when things are heading south so you can get ahead of the problem.  Monitoring tells you if there is an outage so you can fix it, sometimes even before users notice.  Monitoring reassures you when there is not an outage so you can sleep at night.

Over the past two decades, I believe as a company and as an industry we have done a good job of building monitoring to cover routers, switches, and wireless gear.  That's great, but virtually every network today includes a sprinkling of firewalls, load balancers, and maybe some web proxies or WAN optimizers.  These devices are few in number, but absolutely critical.  They're not simple devices either.  Monitoring tools have not done a great job with these other devices.  The problem is that we mostly treat them like just another router or switch.  Sure, there are often a few token extra metrics like connection counts, but that doesn't really represent the device properly, does it?  The data that you need to understand the health and performance of a firewall or a load balancer is just not the same as the data you need for a switch.  This is a huge visibility gap.

Network Insight is designed to fill that gap by finally treating these other devices as first class citizens; acquiring and displaying exactly the right data set to understand the health and performance of these critical devices.

Network Insight for Cisco ASA

Network Insight for Cisco ASA is our second installment in the Network Insight story, following Network Insight for F5.  As you saw with F5, Network Insight for ASA takes a clean slate approach.  We asked ourselves (and many of you) questions like:

  • What role does this device play in connecting endpoints?
  • How can you measure the quality with which the device is performing that role?
  • What is the right way to visualize that data to make it easiest to understand?
  • What are the most common and severe problems that occur with this device?
  • Can we detect those problems?  Can we predict them?

With these learnings in hand, we built the best monitoring we could from the ground up.  Let's take a look at what we came up with.

Access Lists

ACLs define what traffic is allowed or blocked.  This is the most essential task of the firewall and monitoring tools generally don't provide any visibility.

The first thing we found here is there's no good way to get all of this data via SNMP.  We have to pull the config and analyze it.  For that reason, we handed this piece off to the NCM team to work on.  Check out more here: Network Configuration Manager 7.7 is now generally available!

Site to Site VPN

Site to site VPN tunnels are the next most important service that ASAs provide.  They are often used to connect offices to data centers, data centers to cloud providers, or one organization to a partner.

Yesterday, you could monitor these tunnels by testing connectivity to the other side of the tunnel, for example an ICMP monitor to a node that can only be reached through the tunnel.  Today, we poll the ASA itself via SNMP and API to show a complete picture including:

  • What tunnels are configured?
  • Are my tunnels up or down?
  • If a tunnel is up:
    • How long has the tunnel been up?
    • How much bandwidth is being used by the tunnel?
    • What protocols are securing the traffic transiting the tunnel?
  • If a tunnel is down:
    • How long has the tunnel been down?
    • What phase did the tunnel negotiation fail at?

Site to Site VPN Tunnels.png

This means we automatically detect and add VPN tunnels as they're configured or removed and constantly keep an eye on these very important logical connections.  I'll highlight a couple interesting things.

Favorites

We're introducing a simple new concept called favorites.  Marking a tunnel as a favorite by clicking the star on the right does two things.  First, you can filter and sort based on this attribute.  The page by default shows favorite tunnels first, so you will always see your favorites first until you change the sorting method.  Second, it promotes that tunnel's status to the summary screen.  We found for most ASAs there were a couple of VPN tunnels that were wildly more important than all of the other tunnels.  Here at SolarWinds HQ for example, it's the tunnel to the primary data center.  At the primary data center, it's the tunnel to the secondary data center.  Favorties provide a super easy way to add extra focus to the tunnels that are so important that a big part of the story of the health and performance of the ASA is the health of the tunnels themselves.

Tunnel Status

What is the status of the tunnel?

Turns out this is a harder question to answer than it looks.  Tunnels are established on-demand.  If you just configured a tunnel, but have not sent any interesting traffic so the tunnel is not up, should we show it as down (red)?  That doesn't seem right.  What if the tunnel was up for 3 months, but interesting traffic stopped coming so the tunnel timed out and went back down, but is prepared to come back up as soon as interesting traffic is seen?  The tunnel is definitely "down", but should it be red?  Probably not!  We spent a lot of time thinking about this and talking to you guys to determine the logic that decides if an administrator considers a tunnel down, up, or something in between.  All of that logic is built into the statuses you see presented on this page.

Phases

For years, my first troubleshooting step on a tunnel that was down was to review logs and find out what phase negotiation failed at.  This tells you what set of variables you need to review for matching against your peer.  I'm very pleased that this first data point is now right in the monitoring tool that identified the tunnel as down to start with.  I hope it helps you guys get your tunnels back up faster.

Remote Access VPN

When users connect to the office using a software VPN client on their laptop, Cisco calls that Remote Access VPN.  As with Network Insight for F5, we are careful here to use the same terms as the manufacturer so it's easy to understand what we're talking about.

Again, we have to use both SNMP and API to get all the data we need to answer the following questions:

  • Who's connected?
  • Who tried to connect in the past, and what was the result?
  • How long have they been connected?
  • How much data have they uploaded and downloaded?
  • What is their session history?

Remote Access VPN Tunnels.png

Again, I'll highlight a few things.

List View

One of the challenges is the sheer number of remote access connections there are.  We know we do not do good enough job at dealing with very large lists today and our UI Framework team has been working on solving that.  This page is one of the first implementations of the new List View that they created.  This list view gives you the tools to easily deal with very large lists.  The left side of the screen lets you filter on anything shown on the right.  The filters available are considerate of the data and values seen on the right, so we don't have useless filters.  You can stack several filters and remove them individually.  Finally, after filtering your list you can still sort and search through those filtered results to further hone your list.

Remote Access VPN Tunnels - Filters.png

You'll see this list view a lot more as time passes.

Interfaces

Whereas interfaces are the main story on a switch or router, they're an important secondary story on an ASA.  We rebuilt the interfaces view from the ground up based on the List View.  Along the way, we made sure we were building it for a firewall.

Interfaces.png

NAMEIF

As my fellow ASA Administrators know, nameif is not a typo.  Nameif is the command you use to specify the name of an interface on an ASA.  A nameif must be configured for an interface to function, and from the moment you specify the nameif onward, every other element in the interface references the nameif.  ACLs, NAT, you name it.  In other words, the identity of an interface on an ASA is its nameif (like CPLANE or OUTSIDE), not it's physical name (like GigabitEthernet0/2).  Accordingly, that is the primary name shown here, with the physical interface name shown only if the interface isn't in use and doesn't have a nameif.

Access Lists

If you have NCM to pull access lists from configs, we will identify which access list is applied to each interface and provide a link to review the access list.  This is super convenient in practice.

Security Level

Security levels have some control over what traffic the ASA allows.  It also provides a quick indicator of how much the administrator trusts the network connected to a specific interface.  Kind of important things for a firewall.

Favorites

Again, we're using the simple favorites concept.  I expect a lot of ASAs to have the interface connected to the Internet favorited!

Platform

All of the things described above are technology services that are built on a platform.  The platform must be healthy for the services to have any chance of being healthy.  The platform sub-view helps you understand the health of the platform.

Platform.png

High Availability

While high availability is a feature of many platforms, it seems to be particularly popular on ASAs.  Additionally, it seems Administrators have to fiddle with it a lot.  Administrators have to failover to perform software upgrades, some choose to failover to change circuits, failover to upgrade hardware, failover for all sorts of reasons.  While I'm concerned we are all using failover so often, it is clear that NPM has to provide great coverage for H/A.

In speaking with lots of ASA administrators we found several different behaviors.  Some administrators were unaware of whether their ASAs were really ready for failover or not.  Some check manually every once in a while, but have had an active ASA go down only to discover failover could not occur.  Some expert administrators were checking failover status, but were also checking the quality of failover that would occur by verifying configuration synchronization and state synchronization.

Our H/A resources takes the best practices we found were being manually used by expert administrators, automates the monitoring of them, and presents simple conclusions in the UI.  If everything is green, you get simple checks and a phrase explaining what is healthy.  If something goes wrong, you get a red X more verbose explanation.  For example, if the standby is ready but the config is not in sync, failover can occur but the behavior of the firewall may change.  Maybe your last ACL change was not copied to the standby, so it doesn't apply if there is a failover.  If standby is ready but connection state information is not synced, failover can occur but all of your users will have to restablish their connections.  Not good!

Of course you can alert on all of these things.

Connection Counts

Firewalls store information about each connection that is actively flowing through them at a given moment.  Because of that, there is a limit to how many concurrent connections they can handle, and this is one of the primary values used to determine what size firewall you need to buy.  It's obvious then that it should also be a crucial part of how we understand the load of the device in addition to RAM and CPU, so we've included it here.

Aggregating connection failure rates is an interesting way to get an indicator that something is amiss.  Perhaps your firewall is blocking a DDOS or maybe a firewall rule change went awry.  Watching this one value can be a leading indicator of all sorts of specific problems.

Summary: Putting it all Together

If we've done our job, we're providing comprehensive coverage of the health and performance of an ASA on all of the sub-views.  Now, we pull all the information together and summarize it on the Summary page.

Summary.png

Details Widget

One of the things that really weighed down the Node Details page for most nodes was the Details resource.  This resource has historically been a catch all for lots of little bits of largely static data users have asked us to show on this page.  The problem is that it kept growing and eventually took up nearly half the page with data that actually wasn't that commonly needed.  Here we have rebuilt the resource to focus on the most important data, but with the additional data available within the "other details" drop down.  This also allowed us to move away from the archaic pattern of Name:value pairs in our UI.  Instead, we describe the device as your peer would.  You can see how the resource reads more like "this is <hostname>, the <context name> context on a <hardware model> running <software version>".

Also, did you know that what we called "resources" in the previous UI framework are called "widgets" in the new UI Framework?  There's your daily dose of useless trivia!

PerfStack Widget

Did you notice it?  The Load Summary and Bandwidth widgets on this page are powered by PerfStack charting.  Try clicking around on them.  It's oh so pleasant.  More to come on this later.

Favorites

The Bandwidth and Favorite Site-to-Site VPN widgets display information about the components you identified as your favorites on the other pages.  I think it's about time we recognized that all VPN tunnels and all interfaces are not equally important.  Some are so critical that their status alone is a big part of the answer to the question: how is the firewall running?  Favorites makes it easy to give them the attention they deserve.

Setup Network Insight for ASA

To get this visibility in your environment, jump on over to the customer portal to download the new version.  After upgrading your NPM instance, the new ASA monitoring should "just work," but here's the specifics just in case.

Already monitoring ASAs?

The new monitoring will start up automatically.  Give the new version a couple minutes to poll and jump over to Node Details for one of your ASAs.  You'll get a bunch of new information out of the box.  For complete coverage as seen in the screenshots above, you'll be prompted to edit the node, check the "Advanced ASA" monitoring check box, and enter CLI credentials.  Make sure to look at the sub-views (mouse over to the left)!

There is one caveat.  If you've assigned a custom view to your ASAs, we will not overwrite that!  Instead, you will have to choose to manually change the view for your ASAs over to our new view.

Adding a new ASA?

Simply "Add Node" and select the "Advanced ASA" monitoring check box on the last step to enter CLI credentials.  That's it.  Give it a few minutes and check out the Node Details page for that ASA.

Conclusion

That does it for now.  You can click through the functionality yourself in our online demo.  I'd love to hear your feedback once you have it running in your environment!

101 Comments
Level 10

Can someone who have installed 12.2(BETA or GA) confirmed if TLS 1.2 only(with TLS1.0/TLS1.1 disabled) is working?

MVP
MVP

Nice a lot of new functionality. I will be installing next week - just completed my change request.

Level 10

We do not have NCM installed, will all of this still work with only having NPM 12.2?

Level 15

Everything will work except where I explicitly called out NCM: Access Lists.

Level 7

What privileged level will the ASA credentials need to be?  Can this be locked down to Read only access?

Level 15
Level 10

I just upgraded our environment using the new installer.  WOW!  That was a breeze! No issues at all. Kudos to the team who put that together!

Only comment would be to make sure you upgrade to 4.6.2 MS .net... install flops if you dont have it and then complains you need it. 


I am REALLY looking forward to digging into the ASA insight.   Our organization has been really wanting something like this and we had no solution available.  Pleasant surprise to see this nugget available!

Level 7

Is there any tips why my Remote Access VPN tunnels - Sessions view is in "no Data to show" state (there is about 83 active AnyConnect vpn sessions). Everything else is ok (example site-to-vpn are shown correctly).

But have to say, 12.2 is great and upgrade went flawlessly!

Level 15

tero.kiminki​, if you're getting everything else then both SNMP and API/CLI polling are working.  Sounds like it's time to call in Support!

I'm glad the upgrade went well.  We've put a lot of work into improving that.

Level 9

Just upgraded to 12.2 on our dev environment.

Is there a way to bulk edit favorite Interfaces?  we have our critical interfaces labelled for easy reference, but I don't really want to walk through hundreds of interfaces to make them favorite if a bulk edit is available. 

if not through the UI, is there a script that can be run?

Level 15

Yes, a SWIS verb can be used to set single interface as favorite, interface ID as the parameter:

Orion.ASA.Interfaces.SetFavorite

A script could loop through a batch.  More details in the SDK: Orion SDK Information

Level 11

I upgraded to NPM 12.2. and NCM 7.7 and I really like the features that it provides especially with the ASA devices. I can now look at the ACL lists and can see where issues might occur due to misconfigurations on the firewall rules. The problem that I am looking at is that I don't see the site to site VPNs, which are critical to the site. I have looked at all the documentation and nothing has fixed this issue. I have a case open with tech support right now but it seems to have little traction. Like tero.kiminki, the SNMP and API/CLI polling are working fine. Anyone have any ideas that I can try while I wait for tech support to respond?

Level 15

hoppingubu​, can you PM me your case number?

MVP
MVP

The pre-flight checker has a nice look.

The cool "wait for it" animation and a ticket of the number of checks remaining along with a progress bar.

SystemCheck.png

Level 9

Thanks, I'll look into that.  I was hoping it would be in the UI, maybe under manage nodes -> interfaces filter.  quickly search for interface terms and select favs like in manage reports section.

One not on updater, It seemed to check for already running SW module after it installed some of our software. I think I had Log Adjuster open for troubleshooting a different issue.  My thought is all pre-checks for running SW modules would be done at the start of install.  Overall, it went smoothly.

Product Manager
Product Manager

good feedback thanks

MVP
MVP

The new install saved me a TON of time.

We use several modules and generally, I have installed one, rebooted the servers, installed the next, etc. And, yes, I know that takes extra time, but with this installer, I was able to do the updates in a couple of hours total. This includes running the installer on the main server, manually updating NTA on its server and running the update on my remote poller. Very pleased with the new installer.

I do wish I could get that animated "wait for it" as a cursor option on my machine. It's kind of satisfying to watch a machine try to solve the "cube" and still end up with mixed color sides.

Product Manager
Product Manager

that's awesome!! very glad to hear that.

Level 9

Good write up. I just finish my upgrade yesterday to 12.2 some weird issue after upgrading the NCM 7.7 the main POLLER stopped working but recover after 12 hours. Don't know what happen it recover itself. Glad that when I'm wake up this morning new version was functional. Observing  

Level 9

I experienced that I cannot find the ACL's until upgrading to NCM 7.7

Level 9

Really like new ASA feature set.  2 comments/suggestions:

1.  On my ASA, the Remote Access VPN Tunnels are showing up 2 or 3 times.  The amount of uploaded and downloaded data for the end user is identical, so I'm quite sure it really is 1 session that's showing up multiple times.

2.  For a feature request, it would be great to see the IP address of the Remote Access VPN Tunnels.  Our HelpDesk gets calls from remote users, and trying to walk the end user through finding their currently assigned IP can be challenging.  It would be cool to see that IP information through this Orion interface.

Thanks!

MVP
MVP

Agh import sample map stuck on main poller why is it always me!!

installler great config wizard same as always... grrr

MVP
MVP

pastedImage_0.pngpastedImage_1.png

re ran config wizard a few times.

deleted site from iis, renamed website folder

on hold to support at the min

MVP
MVP

IPAM didn’t uninstall the old one so both were there new and old.

gotta love support went straight there.

now Config wizard

MVP
MVP

Now it says can’t access a dll   ‍♂️

Product Manager
Product Manager

What version of the installer are you using? Can you right click to show details on the download?

MVP
MVP

Just on with support hoping this is the last Config as still have additional poller and web server and nta database to do

1.2.0.469

MVP
MVP

And then reconnect storage manager and vman as had tls issues with them 

MVP
MVP

it worked but they forgot to reinstall ipam so done and now another config on main poller its 23:06 and this upgrade has been going since 16:30 doh!

MVP
MVP

Just to update this all working now except for SRM but this wasnt working correctly before the install.

loving the new features filter manage entities Search on the main bar ASA access all added!!

Level 9

One other comment on the 12.2 upgrade.  We use a Network Atlas map on our main landing page for Orion.  Before the update, I'm 99% sure that we were showing link utilization/stats on the router connections.

After the 12.2 upgrade, I have a number of "red dotted lines" on the map, which are saying "no connection between nodes."  (I know for a fact that those red dotted lines were not there before.)  I was able to implement a workaround that I found in various threads here, which basically was to edit the map and "Don't show additional info."  However, with that customization, now my Metro-E connections are also showing no link utilization/stats.

When I was on version 12.1, our map worked, even with the MPLS links on it.  Anyone have any idea what changed?

Product Manager
Product Manager

Without more information, it'd be hard to say what's going on. This is not a known issue. Please file a support ticket.

Level 9

Serena,

I already have filed a support ticket (#1283404).  However, that case is considered "resolved", with the solution being that I had to edit our Network Map to "Don't show additional information."  This map edit/customization was not required for version 12.1.  So again, it seems like we've taken a step back in this department.

Level 12

Hi cobrien​,

We are planning to upgrade our existing versions to latest available. We have NPM 11.5.3, SAM 6.2.2, NCM 7.4, NTA 4.1.1 & SRM 6.2. We would like use the new Orion  installer. I have below queries, please answer.

  • Can we use the Orion installer to upgrade from the above versions to latest versions?
  • If we can run one installer, would it install to the latest available version of each module?
  • Before executing the installer, we need to install the .Net 4.6.2 on both pollers (we have 2 pollers), primary and additional?
  • Does the same installer can be used on additional poller or is there any other installer that needs to be executed on additional poller?

Looking forward for a quick response. Thank you in advance.

MVP
MVP

Anyone having issues with alerts and sql or swql coming out with

MACRO SQL ERROR - Object reference not set to an instance of an object

the below worked before the upgrade

${SQL: SELECT substring('${N=SwisEntity;M=Volume.Caption}',1,3)} on ${N=SwisEntity;M=Volume.Node.Caption}

MVP
MVP

aghh hotfix to install!!!

Level 9

Cisco ASA Features don't work with IOS 9.1 - vpn's do not shown!

Level 12

I asked basically the same question in a demo for 12.2 two weeks ago and was told, like you we could got to NPM 12.2 directly from 11.5.3. The tech from Ireland stated yes, but would still like to hear the answers to your full questions as well.

Level 9

Quick tip on the Access Lists after the upgrade:

I had to force an NCM configuration pull and grab a current running config in order to have the ACL tab appear in Network Insight for ASA - even though NCM already had current configs in its repository.

Just a tip in case anyone else faces the issue I did.

Level 10

Is there a way to bypass the CPU requirements check?  we have 8 CPU's assigned but they are each 2.0Ghz running in VMware.  we realize that upgrading would be better but we would like to use some of the newer features before we buy the newer blades.

MVP
MVP

installed hotfix that says it will fix the above ran the config wizrds on main poller, additional poller and web server.

Still doent work??? what did the hotfix do.

sent diagnostics to support hoping for a call back on ticket #1294415  tonight as not in tomorrow and could do with this being fixed

MVP
MVP

just click next its just a warning.

Its only if there a licensing issue or something broke you cant continue.

Level 9

Great write up.  Network Insight is particularly interesting though I'm looking forward to this being available for other platforms as we have very few ASAs ourselves.

I'm hopeful Network Insight is written in such a way that makes it easy to adapt to different platforms.

Perhaps even a "Generic Firewall" template that could be used for those devices not yet defined allowing admin to define the relevant OIDs and transforms to derive ACLs, VPNs, Interfaces, Platform information etc.

Though this might mean some features are unavailable or restricted but could help significantly with basic monitoring of such devices and might make development of official templates easier.

Level 7

I upgrade NPM 12.2 already(09/25/2017), but i found many problem.

- ASA monitor is not working error : Plugin "ASA Business Layer" failed to start on [xx]. Please restart the Module Engine service on [xx].

2017-10-06 19_28_01-Node Details for ASA - Summary - FW-5H05A-1.true.th.jpg

The group members changed issue. In picture that time group doesn't gained any member but event show the event from past.

2017-10-06 19_29_43-Network Summary.jpg

- CPU utilization issue. normally the cpu usage around 10-60%, but in 2-3 days after restart npm server, CPU will increase to 100% and never drop, so I need to stop solarwinds information service v3 to decrease the CPU usage.

2017-10-04 12_52_30-Greenshot.jpg

2017-10-04 09_19_51-Greenshot.jpg

2017-10-04 09_19_18-Greenshot.jpg

Level 7

- Average response issue. from picture below, I already upgrade NPM(09/25/2017) after that the average response time will increase every day. On Sep 30, I need to restart server NPM because the npm detect node down(all node) cause from the higher response time. after I restart it, average response is back to normal. And this issue happen again on today.

2017-10-06 19_53_45-Node Details - Summary - CRT01_5H07.jpg

this node is the same issue.

2017-10-06 19_54_03-Node Details - Summary - CRT01_AG08.jpg

- after that I try to move some node to APE(10/03/2017). and the problem is solved. I'm not sure what is the root cause.

2017-10-06 19_54_39-Node Details - Summary - CSW_5H07_1.trueidc.co.th.jpg

- Solarwinds information service v3 force stopped by itself.

please help me to investigate the problem, from now I need to restart server every 3 days to solve this issue.

Product Manager
Product Manager

You need to go to NPM 12.0 for the 12.2 installer to directly go to the latest. If you run the installer, it will show you step upgrade path to get to 12.0. This will also be the case for your other products that are on the older version. There is a minimum version for this installer to be used for a single consolidated install.  Once you meet the minimum version requirement, you can use the 12.2 installer to get every single product up to the latest in one installation.

.NET 4.6.2 should be on your primary & additional poller.

You will need to use the scalability engines installer for your additional poller, not the NPM 12.2 installer.

Product Manager
Product Manager

No, you need to get to NPM 12.0 at least before using the new installer.

Product Manager
Product Manager

Exactly correct!

MVP
MVP

Our install went really well and was way, way, way faster than installing individual pieces.

Very impressed.

Level 15

Looks like something went awry in the upgrade to be having this many problems.  We'll need to take a look at your system and walk through some troubleshooting steps to ensure full resolution.  Could you call in to Support to start this process?: SolarWinds Contact Us

About the Author
Lifelong technology enthusiast. Network Engineer turned Product Manager for network products. By geeks, for geeks! I started my career as a call center agent at a wireless ISP. I moved into the Network Operations Center to operationally support their network. I moved to another company to be a Network Engineer, and fulfilled that role at several different companies in different verticals including Healthcare, Software, and Finance. Eventually, I found my calling as a PM, where I work with all of the functions of a business, and particularly Development, to determine what to build next to deliver the most value to our customers.