Funny! But, it is rather scary...
Our lab environment is not as large nor can it represent our production environment. We have had success in leveraging GNS3 to better represent our production environment to work through changes prior to production deployment maintenance windows.
As long as you're not updating configs on your production equipment, testing these in production shouldn't be a big deal...you can limit your devices as well
Just firing up a GNS3 instance to test against, as we speak.
I take it you have GNS3 and NCM running on the same machine? Or, are you bridging GNS3 to the NIC of another PC that NCM connects to?
We have customers looking at our Orion system, so we are running it as a dedicated VM.
I personally wouldn't run it on the same system, unless you are in test&dev.
Compliance policy reports are typically fairly easy to test. The fun is testing Config Change Templates and figuring out what the variables will accurately output. To assist with that I created a Cisco Embedded Event Manager (EEM) scriptcalled echo. Then I just create a change template that runs the script followed by the variables I want to test. To run this EEM script you would use the command event manager run echo <Insert_Variable>. I am doing this for each variable I want to test. There are likely some problems with it, but I just created it this week when I got frustrated with the lack of change template variable documentation and examples.
Another idea would be to enable 2 step change authorization for all users, to see the exact output of the scripts and to be able to schedule remediation that isn't a change template.
event manager applet echo
action 00.0 syslog msg "Count: $_none_argc"
action 00.1 if $_none_argc gt "9"
action 00.2 syslog msg "Too many args for this script"
action 01.1 elseif $_none_argc eq 9
action 01.2 syslog msg "Output: $_none_arg1 $_none_arg2 $_none_arg3 $_none_arg4 $_none_arg5 $_none_arg6 $_none_arg7 $_none_arg8 $_none_arg9"
action 02.1 elseif $_none_argc eq 8
action 02.2 syslog msg "Output: $_none_arg1 $_none_arg2 $_none_arg3 $_none_arg4 $_none_arg5 $_none_arg6 $_none_arg7 $_none_arg8"
action 03.1 elseif $_none_argc eq 7
action 03.2 syslog msg "Output: $_none_arg1 $_none_arg2 $_none_arg3 $_none_arg4 $_none_arg5 $_none_arg6 $_none_arg7"
action 04.1 elseif $_none_argc eq 6
action 04.2 syslog msg "Output: $_none_arg1 $_none_arg2 $_none_arg3 $_none_arg4 $_none_arg5 $_none_arg6"
action 05.1 elseif $_none_argc eq 5
action 05.2 syslog msg "Output: $_none_arg1 $_none_arg2 $_none_arg3 $_none_arg4 $_none_arg5"
action 06.1 elseif $_none_argc eq 4
action 06.2 syslog msg "Output: $_none_arg1 $_none_arg2 $_none_arg3 $_none_arg4"
action 07.1 elseif $_none_argc eq 3
action 07.2 syslog msg "Output: $_none_arg1 $_none_arg2 $_none_arg3"
action 08.1 elseif $_none_argc eq 2
action 08.2 syslog msg "Output: $_none_arg1 $_none_arg2"
action 09.1 elseif $_none_argc eq 1
action 09.2 syslog msg "Output: $_none_arg1"
action 10.1 elseif $_none_argc eq 0
action 10.2 syslog msg "Input is missing"
action 99.9 end
I think you are getting close to what I want to do, but I'm not an EEM expert.
Can you explain a little more about your 2nd option? Can you have NCM output the change script without pushing it to devices? I apologize if this is obvious, but my only access to NCM is in production which mitigates my ability to play with it much. I've only just started looking at all this.
if you create a config change template (not just a simple script or remediation CLI) it should display a page that has a preview of the commands to be ran. The second option is enabling NCM's Change approval system (under NCM settings) You get 3 options for the Approval mode when you run the setup wizard for it. One-level, Two level, and two level for all users. if you choose the two level for all users it will show the commands to be to ran in the Request details section. I have seen the approval system be a little bugging when I am level 2 approver and also the submitter, but I haven't spent much time testing. Also the script previews are not editable.
Going to the EEM script. all that it basically does it echo the responses. It looks so complcated because EEM treats each item after the script name and separated by a space as a different variable. Also if you try to use the variable and it isn't there it will error. Since I don't know if there will be any spaces in the NCM variables I accounted for 8 spaces using the script.
If I made things more confusing please let me know, my sinus meds are taking a toll on me today and I can clear it up on Monday if need be.
I like the GNS3 option, but I had not set that up yet. In my last job, we had several "shelf-ware" switches that I tested all config scripts, IOS upgrades and policy templates on.