Create alert for the monitoring of Windows service running in clustered mechanism on 2 servers

I am having a service which runs in clustered fashion on 2 servers and is active on 1 server only at a given moment of time. I need to trigger alert only in case when the service is down on both the servers. If its down on one server and running on another server, i dont want alert to be triggered. I have created below condition but its not triggering the alert as I tested for one common service which was in stopped state on both the servers.

Find more posts tagged with

Service Monitor

sam

windows service monitor

cluster

service

solarwinds

Windows

Accepted answers

All comments

SteveK

I have seen some weird behavior in the past were the alert will not trip unless the monitor was at one point in an up position. So you maybe need to start with something simple like print spooler that you could hopefully start/stop at will.

I'm curious if SolarWinds groups would be better for something like this. I haven't personally tried this out yet, so the following is all in theory.

So Settings -> All Settings -> Manage Groups

Build out the group, but make sure you select Advance and then Status Rollup mode to "Show best status."

Then on the Add Object screen, set it to application and add the monitor from the 2 nodes (or go dynamic if its better for your needs).

I think your alert would look something like:

I would also make sure you have at least a couple minute time threshold in place to avoid false alerts (probably 5 minutes). With SAM polling defaulting to every 5 minutes, you could theoretically have a temporary down/down situation based on those cycles as well as a the group refreshing every minute.

vikasp

Thanks for your reply, but i guess this wont work in my case as I am having services under one application. And All 4 are running in clustered mechanism.
Using the group feature, I would need to create 4 separate groups.

adam.beedell

The reason this one doesnt work is that it's evaluating the condition per entry in the nodes table, there will never be a node with 2 names

There's the checkbox for advanced alerts below that would work but is a little hard to understand

Groups is easy to understand but a bit labor intensive

I've wrote a SWQL one utilising right joins in the past which groups without making groups, it feels a bit cursed though.

SteveK

So going at it from a purely Node/Application/Component side. Your alert object would be component. I would then build out the scope to include both the component name and application name. Then your trigger condition would need to be the advanced block with the different groupings for component down and node name is X. The main thing I don't like about that approach is if you add/remove nodes from the cluster, you would need to update the alert. In my experience SW doesn't have the best searching for alert criteria, so it could get messy quickily.

I would still lean towards the group side of things, at least for a POC. The only thing that would need to be changed is switching it from an application based group to a component based group. If you are needing this for 4 seperate services, then you would need 4 seperate groups.

You could use Dynamic groups as well:

As @adam.beedell mentioned below groups can be a little more labor to get setup, but I think it would offer better flexibility in the future. Plus depending on the amount of alerts in the environment and how things are documented in your environment, maybe a bit cleaner.

adam.beedell

Looked up how I did the groupless group one last time

Obviously this isnt on the right table, but same thing would work in theory

Idea being you RIGHT JOIN grouped data using the group verb, if both are down in the group and your node matches against the broke stuff it goes off.

You'd have to think about how to define what your cluster is vs some other instance of those monitors.

shuth

I've done something similar in the past with by adding in the cluster IP as a monitored node.

Server 1 - Monitor the services but mark them as Not Running instead of Down.
Server 2 - Monitor the services but mark them as Not Running instead of Down.
Cluster IP - Monitor the services, mark as Down

You can use the monitors on Server 1 and 2 to see which one is running the active server. Changing the service to show as Not Running instead of Down mean they won't go red.
The cluster IP should always be pointing to whichever is the active server and have an active service - if it goes down, theoretically both services are stopped.