Welcome to SolarWinds Lab, and a special mega wrap-up of all the Orion Platform updates throughout 2019. This is actually part two of a series focusing on enhancements designed to make the lives of system administrators and application engineers easier. So first, from a third of the way around the globe, in Cork, Ireland, please welcome my fellow Head Geek, Sascha Giese.
Hey there to everyone including the elders of the internet and I'm so glad that we broke all the content into two different sessions because there's so much stuff to talk about. And I hope that everyone watched the previous episode.
Yeah, definitely, and check out our homepage, you'll find part one there. And then of course, somebody that you all know from previous episodes of SolarWinds Lab, multiple years of THWACKcamp, THWACK forums, SWUGs, ask-me-anything events, webinars, and more, Senior Product Manager, Server & Application Monitor, and all-around guru, Serena Chou. Thanks so much for being here.
Thanks for having me, Patrick. I'm really glad that we broke the end-of-year highlights into two parts as well.
There was a lot.
There's been a ton of updates that system admins have been asking for, and we'll definitely have some time to do some deep dives today. The feedback from THWACKcamp attendees and also at the Microsoft Ignite booth was awesome. And really, I think that those customers will get a lot out of the episode that we're shooting today.
And it seems that all the updates focused on the network portfolio is about performance, not only the performance of the web platform, but also usability improvement.
Oh, absolutely. But remember that SAM is an Orion module, so we also get the benefits of that on our platform.
So then basically, all the system module updates are just buttery goodness on top of that?
I don't think of it exactly like that, but remember, if you're watching the What We're Working On on the THWACK forum, you probably have noticed that we've definitely been focusing on performance enhancements, but that's not limited to the web console, it also applies to helping system admins save time by making configuration easier, so yeah, that's what these two episodes are about.
So what would you want to highlight about Server & Application Monitor?
SAM has been the go-to monitoring tool for Office 365 and Azure environments for a long time, but we got some really great ideas and feedback from the community about how to make it even easier. So today I'm going to show you how to apply some Azure out-of-the-box templates that we've provided to save time getting metrics to measure.
And community's a great keyboard here. I'm going to show Server Configuration Monitor and we finally brought to the product a long-standing feature request, Linux support, and it's well worth the wait.
Yeah, just a few people have been asking for Linux software change moderating for a while. That's going to be great, and of course, I'm going to geek out, well you can probably guess, I'm going to geek out on APIs but maybe not in the way that we've shown before. Because with the 2019.4 release instead of Orion Platform just being a source of data from its API, it can now actually pull data from other APIs' metrics.
Oh, so you and Leon can finally rip up all those crazy script monitors you've been working on for these last few years.
There's a time and a place for everything. Yeah, so save time, optimize performance, back end, front end, pollers, whatever it takes. Do you think they get the Mr. Mom joke?
I think so.
OK, let's save time. Serena, what about you going to start?
Absolutely, let's get started. Let's look at Azure monitoring templates. A lot of folks at the MS Ignite booth came up and asked me a question to say what are you doing for Azure? I'm moving my workloads into Azure, and I was happy to point them to our release notes that really outline all of the great templates that we've put out-of-the-box for them. But, yeah, let's just take a little deep dive into how you get set up on that. So first, before you start monitoring Azure, you're going to have to start in the Microsoft Azure portal. You're going to need to get set up with some prerequisites before you get started.
What, you mean Azure isn't just wide open and you just point directly at it and start pulling data without any authentication or IAM or anything else?
Yeah, you know, I'm sure there's some security-minded folks who'd really take exception if you could just point at it and start monitoring. So, there seems to be a reason to the madness. But, the prerequisites really involve you going ahead and first making sure you have something to monitor. As you can see in my Azure portal, I have an App Service, I also have a Batch account set up, this is very temporary. I also happen to have a deployment of SAM that I deployed from the Azure marketplace, but first you're going to want to take a look over at your Azure Active Directory,
And this is all in the release notes and the guide
Oh absolutely, if you go to the Getting Started documentation that we have in the Success Center, they'll walk you through this. We have a ton of documentation around that.
Everything is in the Success Center these days.
Absolutely. But make sure that you go to your Azure Active Directory, take a look and see that you have everything set up. I have a service account, and make sure that you've set that up. Also make sure that you have the right roles and responsibilities. So whenever I go back, I'm going to take a look at my subscriptions, I'm going to need this subscription key, and then also take a look at your secrets that you've also created in your Azure portal. Once you gather that information, you set up your, we have the right permissions, then we're ready to go. For me, I needed to make sure that I was an owner of my resource group, and so that way my SAM monitoring account could reach all the things that I wanted to monitor.
And that way you're using the resource group to automatically filter it down to a subset of resources that you want to monitor.
Absolutely, and then you can control the scoping on your side. So, everything's set up here. I've got my things that are monitored, I've already gone ahead and made sure that I have all the rules and permissions. The other thing that is interesting is that if you want to monitor using SAM, you'll end up with a nice, applied application template detailing your app service. I'm actually just supplying two of those default templates that I just showed you in the release notes. You can get to that either through the content exchange in the customer THWACK forum that we have for SAM, or you can also reach it through our release notes. There are a number of places that you can actually get to those.
I like the way that you broke these out in terms of requests codes, right? So you can actually break out either something that looks like maybe it's application configures, error, the 404s, or it's maybe security, or it's actually an internal error with a 500 series error code as well.
Well, thank you. I'd like to take all the credit for that, but that actually matches pretty closely to what you can get into Azure today. We wanted to make sure you had that same feature functionality, you wanted to see your data in, data out, your HTTP requests, but you'll notice we actually have a little more granularity than what's out-of-the-box in your Azure portal. So, going back from here, how did I get here? This is all fueled by a template which you would be managing through your All Settings, SAM Settings, Manage Application page. So your template's going to be right here. All you have to do is apply it to a node of your choice. So, I have a freshened solution here, I went to solarwinds.com, I downloaded Server & Application Monitor, and went ahead and downloaded and installed it here. You can see that I'm on the latest 2019.4.
It's so much easier just having the one version now for all,
It just makes things so much easier if you're going to upgrade environments with multiple products. But here, because SAM is the shining star, we have
It is the shining star.
It is the shining star. We have this here, it's, and for those people who are just discovering SAM for the first time, one of the things you're going to notice is that we're already managing the Orion instance as a node already, so if you wanted to apply these templates, you can go ahead and jump over here and say, hey, I want to assign this to a node. I've already assigned this to this node here, but let me just go ahead and remove that, so that way you can get a better view of the process. So, going to delete this real quick. And, so, as I was saying, you can see, I don't have this assigned to any nodes, so I'm not going to see any data for this. But if I walk through, assign it to a node, I already have my Orion instance being monitored today. This is just going to be the node that I apply it to, going to click next, I don't want to inherit credentials from the node because those are not really meaningful. The credentials are actually...
You don't have the same credentials for a physical server that you have for your Azure account?
You must be a system admin's nightmare if you want to set it up that way.
Just use admin for everything, domain admin for everything.
From here, if you go ahead and select credentials from the template, then you're going to either test it out, make sure that you're getting all the data here, but I'm just going to go ahead and assign this application monitor to the node.
I tend to use test a lot, especially if I'm connecting to something new or based on an API, or in this case, where you are going to do a little pre-setup ahead of time, it'll just save you a little bit of trouble maybe that first time you apply one, but theoretically once you've got the credentials set for the first one then they will all just work because they're going to share those same credentials.
Well, so Patrick, I think that you might not have asked me the question that I would have expected since you brought up the security concerns that you have, which is when did I ever apply my credentials? And so, one quick look before we kind of head over to the view of this. You'll notice that underneath the covers, each of these components are actually Windows PowerShell monitor, and so if you wanted to customize this and use this template to go against different services that we didn't provide in an out-of-box template, that's very easy. And if you do that I hope you actually contribute that back to the community because product managers like myself will go to the THWACK community, try and grab those, and see if we can take learnings from what you've tried to build and put that into the product out-of-the-box.
Well, that's a big part of what's different here, right? Is there were monitors that were basically, these are script monitors, right? Just like the Linux monitor or anything else, so there were monitors for this and a couple of other things that for Azure that were previously in the sharing library. But these are essentially now curated, right? These are ones where we've taken the best practices from the community, really tested them, tested them with Microsoft, and they're essentially published by us and supported.
These are definitely supported, we've put a lot of work into these. These are curated, as you said, I love that word, to make sure that you have something that once you just give us credentials, which are essentially script arguments, then once you have that set up, you can apply your templates in multiple nodes and get going on your monitoring. The thing is, I know that you're a fan of tinkering and using APIs,
I know you're going to go into this a little later, but really, I didn't want to take away all of the power of that from folks like you who are what I consider power users because you might need to generate your own custom PowerShell script, in fact you might have PowerShell scripts that you've already generated, and have been using those for monitoring, but you never really knew how to plug that in to SAM to get that streamlined alerting, integration into Service Desk, which we just released as well, but yeah. So, this is the genesis of these Azure templates is that each one is a script that we've custom-written, we've put that template out there, so you can override these, you can edit these, to get the results that you want if you need to have that kind of power. But really, again, we're all about optimizing for you, so these have all been just created out-of-the-box, you can see that it took me a few clicks to start monitoring it, and you can see that the template is applied and we're just going to wait for those components to be monitored.
And they'll start coming in.
Absolutely, and this is a pretty similar structure to what you're going to see for all of our Azure templates. You'll be able to see all of the important node counts, performance metrics that you really need from your Azure portal, but integrated into SAM.
I think you said earlier that this was a pretty extensive library already, but you're continuing to add more and more as new services roll out in Azure, right?
Absolutely, and the thing is I think the point of what I'm hoping that you'll take away from this episode and this demo is that we've already been using the tools that we've had at hand for a long time in SAM to give you that out-of-the-box application template that should save you a bunch of time and as you're doing your Azure monitoring today, but I think once they get to your session and your demo, they'll actually see the future of where we're going with this to make it even faster.
It is kind of interesting watching the Orion Platform develop for all of the years that I've been here, because a long time ago there was a sort of preconception that SNMP was the way that all things were monitored, but we're a software company, we would much rather just use an API to get to things, so increasingly it almost becomes glue between inbound and outbound APIs.
Absolutely, and one last thing to show you, just because Tom LaRock was kind enough to give us his credentials, don't tell him that we're showing his Azure environment.
Microsoft MVPs have superpowers.
We also took it one step further for those of you who were wondering what's going on with the cloud monitoring. I do want to point out that that SAM server that I deployed from the Azure marketplace is also available here. I'm monitoring it and getting all this great insight from it as well. And that just was a matter of applying the right credentials.
So it looks to you like any other physical instance if it was running in VMware, Hyper-V, on-prem, or running in Azure, as a virtualized workload. Just looks like another server.
Yeah, and this is just a proof point that not only can we monitor your Azure with the application templates that I've just shown you, but we can also be deployed there, you have the flexibility to have SAM deployed wherever you need it to be.
Serena, thank you so much for going through this. This was a big hit at Microsoft Ignite and we, our customers, are deploying increasingly either into or monitoring resources in the cloud, and this has taken, we made the joke earlier, this really has taken what was custom scripting or a little bit of fiddling, or spending some time looking at how-to guides to pretty much just point and click and apply. Thanks for showing it to us.
All right. So you actually have a meeting to go to. Will you come back a little bit later?
Yeah, I'll be back.
All right, see you in a minute. All right Sascha, what are we going to talk about? Maybe, I don't know, Linux file monitoring?
All right, so the Server Configuration Monitor, when did we publish it, like a year and a half ago? And one major feature was missing, basically Linux support, OK? If I'm not mistaken, someone suggested it in THWACK in last November, so it took us a little bit of time to get into product, but it's a major change, right? It's basically a new product. Let's have a look. There's one requirement, and that is obviously we need the agent to be installed. That's the bad news. But the good news is it is exactly the same agent as we used it in the past for monitoring. I think Patrick there is just a plug-in that we need to enable for our monitoring with SCM, right?
That's right, and you can check to see if it's already enabled on an existing node, and great point, it is using the same agent. And that's something that's true for both Windows and Linux, that agent is essentially just a wrapper that contains plug-ins, so if you already have the agent installed, it'll just add the plug-in without making any changes to the machine.
OK, so let's quickly go to the settings, we're on the settings already, and in the middle of my screen is Manage Agents, so let's open that one. And we will see all the agents here available in the online demo. And I see here the Red Hat box, Ubuntu box, so let's quickly select that one, and go to More Actions. View installed agents plug-ins. And here it is actually, so it's already installed, SCM agent plug-in. OK, let's go to the main page of Server Configuration. And let's pick the Linux box. So, that's basically everyone who used the tool before with Windows is familiar with the concept, it's exactly the same. So here, right inside we actually see a few Linux boxes, so I've taken a shortcut and click the machine.
So a lot of times you'd actually search for that, right? Or maybe you had something else that would pull that up or you'd pull that up by vendor.
We could, we could, yeah. So, here at the left-hand side we see actually something interesting. We see some of the out-of-the-box templates are already applied. The Linux Essentials, and Security and Permissions. Something is missing. What we don't see here is the hard- and software inventory and the reason for that, let me guess, let me quickly open the summary page. The reason is pretty much as I guessed, we don't have SNMP polling enabled for this node. So for the inventory, we need the agent and SNMP. So I'm going back, and let's have a look at one of these templates here. If you populate the Linux Essentials, there's one major difference here from using an administrator account or non-admin. If I hover over the individual elements, you just get a small description of what it does. As an administrator I would actually see the scripts which would kick off in the background here, which is quite nice. Let's have a look at a change, just a few configuration changes, and there's some nice information.
This is going to give you that diff view.
Oh yes, that is the new view, actually, that is the new view for JSON, et cetera, where it would completely ignore the format and would only show the actual values, and also what you could do here is change encoding in case you deal with different languages, which is quite nice too, because otherwise it might be a little bit scrambled over there.
Does anyone deal with different languages?
Some do, some do. And again, the concept very familiar, like we all use from Network Configuration Manager, so we see the color codes and the change. And if you want to see the full configuration, we could just click it here, but it will actually tell us what has changed right here. And what you also see is this guy. So in the past, we created a baseline, at the left-hand side, is basically the previous configuration, and we probably tagged that as a baseline, and it is a mismatch, so something is wrong here.
So there you're looking for a config drift over time.
Oh yes, basically your golden configuration. OK, what we also see here is Docker, and Docker is actually not an out-of-the-box template. That tells us we can create our own templates, right? And let's try if we can replicate at least a little bit here. So I'm going back to the settings, and again, to the Server Configuration Monitor settings, which is here, and here's the profiles, or templates or whatever we going to call them, and a few of them are already applied. That includes the out-of-the-box, as it happens, they're actually called out-of-the-box, and our own created templates or profiles. So how does it work? We just click Add, and we give it a nice name. This is by the way, I think it's Leon's password, but we can blurry it out later, so now we click Add Element and here's where some magic happens. File is pretty much familiar from well, Windows monitoring, but what's new here is the Linux file system, so we can get our whatever, what's the first one, something with time, whatever, we could add the file information, path and file information but also Registry, PowerShell is not really Linux, but we could just go to Linux, OK, and here's actually, oh this is interesting, if we go to PowerShell again, OK, look here, something is missing, and that's the reason here we could support multiple scripting engines, so we are not limited to PowerShell, so we could go for, I don't know, Bash of course, Titan, Perl, to be honest I'm not sure if there's any limitations as long as there's an integrator, we could just kick it off right from here. And, this one, I think we've seen this in PowerShell how often will the script be kicked off, and how much time do we give it. So nothing new, just now Linux support, which is quite nice. OK, let me go back. If we move on to monitored nodes, there's also a nice new feature so we can easily remove profiles from here, or physically stop monitoring to nodes right from here.
This is really huge, right? To be able to extend change detection across both operating systems.
Well, it is huge. It kind of feels as if it would be a new product, and it's release number four within a year, and the Linux support was requested, and badly needed. It's finally there, which is awesome. So what's left, we should probably talk about available resources if you look at the screen, there's a link, getting started with SCM, there's the Customer Success Center, and there are also lots of other recordings, videos, how-to-use the features, the old features and the new features, but you're getting impatient there, you want to talk about your API pollers, right?
I don't get impatient talking about APIs, I'm always ready to talk about APIs. Yeah, but I think that's a really nice segue here because we already looked at how we're using scripts that are designed to work with the APIs for Microsoft Azure, for example, for the Azure monitoring templates before. In this case, if you were managing Linux environments you are probably working with APIs pretty regularly. I mean, they typically run headless, you're running either a script or APIs to make changes in those infrastructures, so the other thing is certainly we've talked about using scripts before to access data from or to make changes to the Orion environment through its API. But what if you could actually go grab data using the new API poller that's included in the 2019.4 release, so that's what I'm going to show you next.
Sounds like magic.
OK, so the first thing to remember is that Orion at its heart is a data aggregation platform. Once upon a time it was mostly SNMP to do monitoring, and now we're using lots and lots of different APIs. When you look at potentially hundreds of applications under SAM, they could include scripts as well. There is an awful lot of data and it's gathered in a way that's appropriate to the type of data, the type of system that it's connecting to. And of course that's all available to you through the SWIS API to be able to gather that data as well. And the nice thing for those of you who've been working with SWIS, is that you notice instead of going directly to the database, that that API is stable, it doesn't really change over time, so it makes sure your integration is going to be sustainable over time. Well, finally more and more vendors are getting on that bandwagon and making their data available that way too. So one of the things I wanted to show you real quick is the new feature that was released as a part of 2019.4 that allows the Orion Platform to actually go and get API-based data that's coming in from REST queries and pull that in as a metric just like anything else. OK, so I just set this up real quick. It's got a couple of servers in it, and remember you can always do this with eval versions to get started quickly, you don't need a big environment when you start messing with APIs. So what I've got here is a couple of different elements and I'm going to show you one first, and then I'm going to show you how I built it. You'll see this one here that says AppOptics, this of course is an example, I just use it because it was easy because I can grab a ton of data out of AppOptics, and of course, you have all seen screens before like this one where I'm pulling a whole lot of metrics in from Azure, AWS, custom applications, mongoDB, and other apps. So if I wanted to pull that in that's pretty straightforward, because it provides, just like Orion does, an API where I can pull data in. So for example, there's a resource right down here for API pollers, and I've got a couple of them, right? So one of them is I'm getting a count on the number of microservice invocations that are a part of an Azure system, and then I've also got one here for CPU, and this one is for total user CPU utilization on a Linux instance. I'm getting a percentage back there, and if I look at this data, like in the case of the CPU utilization, it's going to open it in PerfStack and I can get all of those values that are coming out of that API. Let me show you how we got that. Scroll up here to the top and it says API poller, and we're going to go to edge CPU user. That's the one I defined earlier, and I'm actually pulling, this is coming from AppOptics, their API URL is api.appoptics.com, this is the REST URL that's documented in their user documentation, and you'll find that most systems that you can gather data from, regardless of the vendor, will have decent documentation, and this is pretty much just cut and paste. I've got a, one thing that I did add here, is I'm using Basic authorization, so I've added a header to this, which is authorization Basic space, and then this is a Base64 encoded standard Basic authentication value, and I'm going to click Send Request. And you can see that I'm going to get back the time series data with the measurements for that value. And then over here on the side, you notice this little button that says monitor this value. When you click on that button, it's going to pull it up here, and then this is essentially the path that's used to extract that value and then render it up to this poller. When you click configure on that, you can set the name and description and a couple of other things, but that's the way that one was created. Now we're green, good. Now I'm going to show you one that's a little bit more interesting. Notice this one here, ambient weather.
Is that coming from your house?
You know that I'm always going to use those APIs whenever I possibly can, but yeah, so we're looking at the wind speed in miles per hour, temperature, whether or not the battery for the sensor needs to be replaced, apparently it does, I'm looking at humidity, and sort of radiation, and feels-like temperature and dew point. So these are environmental metrics that are coming out of the ambient weather sensors at the house, but I didn't want to have to pull that in manually, so how did I get that data? So let's take a look at this API poller definition. And of course we're probably going to have to blur something right here, but across the top, this request URL actually combines the API and application key inside of it so that there's no headers to add. And if I click send request, something very different happens. This is going to come back with one set of data and a whole bunch of other metrics that come back in the JSON payload from that request. They've all been converted over here to values that are monitored. For each one of these I just clicked on it. For example, if I wanted to get, what do I have that's not added here, oh, like event rain, I would just scroll across here to event rain, I'll click on monitor this value, I can say, total rain for this event. I'll click save,
Is that the rain over in Austin or here in Cork, Ireland?
Oh this is going to be here in Austin. Oh, you're right. I should put that in as the title. Let's fix that as a matter of fact. Let's say ambient weather, and we'll say configure, and we'll change this to...Austin. I'll click save. I'll come over here and click save. All right. And so now we've got that volume. So this is actually going to, this will take a minute to poll, but then that will give us the total amount of rain since the last major rain event. Setting this up is pretty easy, it is attached to a node, and for each of these I created a separate node. For ambient weather I picked a static IP, this is not the IP address of the website itself, and I just did that, so I can make sure that it's up because their API I noticed did kind of change around over time. And you can tell when it does that by looking at the data that comes back. When you click on one of these views, it's going to open it up for you in PerfStack, because remember, these are uncorrelated data elements, and I'm getting status and availability and then all of the metrics on these values. Now I don't think you need the status for all of these so what I actually did was built my own view. This happens to be a, what did I call it, Homotops Enviro Data.
So it's really cool to see the relation between these two values.
Yeah, it's really handy, and the way that you'll typically do that, remember, is with PerfStack, you'll want to embed it in another page, and remember to do that, you're going to click on edit page settings, and then just go ahead and add an additional widget. I'm going to search for PerfStack, I'm going to pull all my PerfStack elements and right here, home environment data, I'll drag that over, and I tell you what, I'm going to put that right there. That looks great to me, I'm done adding widgets, I'm done editing, now you can see the data that's coming from the API poller data over here on this side. Here's my charts and graphs for that data, and I don't want to see the last seven days, I just want to see the last data today, and here's all of the data that's actually coming in from that poller. One quick tip, if you see data that looks like this, so you might end up with a situation where the data that's coming back looks like this, you can see that it's an average metric, right, so the data that comes back is OK, but for whatever reason, maybe occasionally the poller simply couldn't find data, or the JSON that came back didn't contain that data element, there's a lot of reasons that might be. You could have throttling, for example, on the API, where maybe you're hitting it more often than you should, and it's automatically returning no data to remind you that you're over the throttle limit. It could be that the data itself just isn't there periodically, maybe the API's going and getting it in real time, and there just was no value that was available, in which case, you might want to ignore that completely. But in general, you'll typically see the data that we saw coming back from that weather API poller, and there are catalogs and catalogs of interesting APIs out there. But in general, data is more typically going to look like we've got here from our sample API. And I mean, it is kind of goofy to use weather, but I wanted to take it out of that domain, of if you're monitoring Kubernetes, or Azure or some other function that is available with the monitoring data and metrics that are coming back from that API, and then we hope that you'll start branching out and start pulling in more and more metrics from systems that you're managing. That might be data that's coming back from cloud API, or Kubernetes, or maybe a custom application, or any number of other metrics that might be provided by the SaaS applications that your environment is working with, be able to pull those in and present those alongside of the rest of your infrastructure metrics that you're using to manage your operation. So that, in a nutshell, is API polling in 2019.4. So what do you think, pretty good, right?
It looks amazing and will open so many opportunities. I'm thinking about attaching my Unify environment, you know Unify doesn't respond so much to SNMP, this will do.
Yes, intentionally, I wish you'd reminded me of that, because I think that's obviously the thing that both of us need to hook up and start monitoring. Because you could use the cloud API to do the monitoring and not have to worry about connecting directly or locally to those systems because it's all manageable through the cloud.
Of course you two would be all over Linux and APIs.
Well, that's what we do, right? And while your Azure templates are pretty cool for getting data, everyone has to admit that playing with Linux and with API, well it's much more interesting playground to tinker around with.
That's right. And of course the new Azure templates basically eliminate what was previously a very geeky configuration, which makes more time for
Other very geeky work.
Patrick, that's just you.
Yeah, that is me. So hopefully between this part two show and the networking part one, you've learned how to increase not just Orion Platform's server and web performance, but also use some of the new UI updates that make configuration easier and faster.
And make sure you check out all the details on all the Q4 updates in the SolarWinds Customer Success Center, also I'd love your feedback on the SAM updates. I'm in the SAM forum on THWACK every day.
And thanks for the comments and the feedback in real time, well, almost real time, by the way, we're talking about time, that's it for now, right?
Yeah, I think that worked pretty well. Latency wasn't too bad, and we got to do this again.
Wunderbar. For SolarWinds Lab I'm Sascha Giese.
I'm Serena Chou.
And I'm Patrick Hubbard, and thanks for watching SolarWinds Lab.