Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Exchange 2007 monitoring with Active/Passive CCR cluster

I’m trying to monitor an Exchange 2007 environment with multiple CCR (Cluster Continuous Replication) clusters of Active/Passive mailboxes together with other servers for Client Access and Transport services and have used Denny’s Exchange 2007 WMI Counters Template (SolarWinds) at together with the recommendations from Monitoring Without System Center Operations Manager at http://technet.microsoft.com/en-us/library/bb201720(EXCHG.80).aspx as recommended in a number of Thwack posts as a basis.

The single servers are reporting and alerting well, but the CCR clusters are proving a bit more trying. All of the servers are monitored in NPM by SNMP & each server has numerous APM templates assigned with various component types. Some of the threshold settings have been tuned, while some have been removed so that they will never raise alerts and these are used for historical tracking only. The CCR cluster servers are also monitored by both NPM and APM. The intention is to give visibility of the service to users (CCR status), but also to track and alert on the detail of their individual member servers.

Some services naturally run on the Active node, but not on the Passive node, so if I test for all of the Active services, the template will alarm on the Passive node, but if I have different templates for the Active node and for the Passive node, they will be fine until a failover occurs when the active-specific services will not be tested on the new Active node which would still have the Passive template assigned to it.

Our 3^rd party mailbox backup solution will only backup from the Active node and will fail from the Passive node so I need to be able to initiate a scripted changeover of the server name in the backup software. I therefore need to notice the failover and to run a script, but not produce an alert.

We run a number of wall mounted overview screens showing overall topology and various production service workflows which engineers can drill down through to the detail of an incident and these all roll-up red on a single failure.

If a CCR Active server fails over to its Passive I need to know

1 that the failover has occurred

2 that the service to the users has been maintained

3 that a script needs to be run to change the server specified in the data backup routines

4 to create an alarm for the broken server to be fixed

5 but not create an alarm, for the failover itself

6 nor for the fact that one member is in Passive & running fewer services

7 and, if using different Active & Passive templates, to swap them over.

I can’t see how to do this. Have I missed something very basic?

SolarWinds Orion Core 2010.1.0 SP1, APM 3.5, IPSLAMGR 3.5, NPM 10.0.0 SP1, NTA 3.6

Find more posts tagged with

Accepted answers

jiri.tomek

Hello Robert,
you are not only user facing this issue. I don't know how Exchange cluster reports failover operation so I can't give you exact answer but I know for sure that APM is not able to dynamically switch templates from one node to another depending on this failover.

Even if you are able to make APM notice failover in cluster, there is no easy way how to switch templates.

I can think of following way to do some of your tasks:

Create only one application template for Exchange and assign it to both active and passive servers. On passive servers disable components for services that don't run there.
Find out how cluster reports failover. If it is for example logged somewhere, use Windows script monitor to check this log and alert on failover.
In alert action for failover execute SQL script that enables components on new active server and disable them on new passive server (you can use some commandline tool provided with SQL server for this).
In alert action for failover execute script that modifies backup program settings.

These steps are only general approach, need further refinement and they depend on ability to detect failover from within APM. If you find out how to do it and if you are interested in more details about enabling and disabling components using SQL, I can provide more details later.

However this is really not standard way of managing APM. I just want to provide you some possible workarounds for thing that APM can't do by itself but it's definitely not officially supported.

All comments

jiri.tomek

Even if you are able to make APM notice failover in cluster, there is no easy way how to switch templates.

I can think of following way to do some of your tasks:

Create only one application template for Exchange and assign it to both active and passive servers. On passive servers disable components for services that don't run there.
Find out how cluster reports failover. If it is for example logged somewhere, use Windows script monitor to check this log and alert on failover.
In alert action for failover execute SQL script that enables components on new active server and disable them on new passive server (you can use some commandline tool provided with SQL server for this).
In alert action for failover execute script that modifies backup program settings.

However this is really not standard way of managing APM. I just want to provide you some possible workarounds for thing that APM can't do by itself but it's definitely not officially supported.

robertcbrowning

Hi Jiri, Exchange 2007 has some running services on the Active server that are stopped on the Passive, so we can use these as triggers. It looks like we'll probably end up swapping over an Active wiith a Passive template in SQL if this can be done simply and safely. If you could let me know how to do this in SQL as a Alert action I'd be grateful. Thanks, Robert

jiri.tomek

Hello Robert,
you should not just swap application templates because this would make your historical data inconsistent. Instead you should only enable/disable specific components on active/passive servers.

If you don't have too many templates that you need to manage this way, it would be easier to enable/disable individual components one by one using SQL. If you have too many servers to manage components one by one, we can come with some more automated way but it would need more complicated script so I would try this first.

Here is stored procedure that enables or disables specific component:

CREATE PROCEDURE [EnableDisableComponent]	@ComponentID int,		@Enable bitASBEGIN	DECLARE @Value AS nvarchar(max)		IF @Enable = 1 	  SET @Value = 'False'	ELSE 	  SET @Value = 'True'		UPDATE [APM_ComponentSetting] 	SET [Value] = @Value	WHERE [ComponentID] = @ROWCOUNT = 0	   INSERT INTO [APM_ComponentSetting] (			[ComponentID],			[Key],			[Value],			[ValueType],			[Required]) 		VALUES (@ComponentID,			'__Disabled',			@Value,			3,			0)END

You should get IDs of components that represent services that don't run on passive servers and then write a SQL script that enables and disables required components on active/passive server. To get component Id just go to component details page and look into URL for "AM:###" where ### is component ID.

Script for specific node being Active can look like:

EXECUTE [EnableDisableComponent] 123,0 GOEXECUTE [EnableDisableComponent] 124,0 GOEXECUTE [EnableDisableComponent] 125,0 GOEXECUTE [EnableDisableComponent] 126,0 GOEXECUTE [EnableDisableComponent] 127,1 GO

First argument is ComponentID, second is enable/disable flag. Example above disables components 123,124,125,126 and enables component 127. Here we assume that components 123-126 are on passive nodes and 127 is on active.

Create SQL scripts for each of your Exchange servers in cluster and in each of these scripts enable services on that server and disable them on the others.

Create Alert on each of these servers that triggers when server become Active. As trigger action for alert add "Execute an external program".

As program to execute use:

sqlcmd -S YOUR_SQL_SERVER\SQL_INSTANCE -U username -P password -i path_to_sql_script_for_specific_server

As you suggested you can use specific service status as Active/Passive trigger. Set condition on specific component coming UP which signalizes that server become active.

Also note that this is really NOT standard way of managing APM and it should be used carefully.

Don't try this on your production database before thoroughly testing it and verifying that it has no side effects on your environment. Also backup your database before implementing this.

robertcbrowning

many thanks. that look great. going to try it out on my test rig asap