This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.
You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

NCM RTCD "Execute an external program" not working

FormerMember
FormerMember


Greetings all:

I've been struggling with this for days.

I've been trying to set up RTCD, but I can't get it to work.  I've completed all the steps in the Administrator's guide, and by a LOT of playing and trial and error, I think I have the issue narrowed down to step setting up the alert to execute SolarWinds.NCM.RTNForwarder ${IP}

I'm running NPM 10.2.2 and NTA 3.8.0 on one server, an additional poller on another, SQL server on a third, and NCM 7.0.1 on a fourth.

On my NCM server, I turned on Realtime monitoring and set all the options.  I then went to "Trap Viewer", "Alert/Filter rules",  and set up the appropriate alert for my hardware.  I set up two actions, "execute an external program" set to run c:\Program Files\SolarWinds.NCM.RTNForwarder.exe ${IP}, and another to simply send me an e-mail so I'd know the alert had triggered.

What I found is that solarwinds.ncm.rtnforwarder.exe never executes.  I can execute it from the command line on that server with no issue, and it works fine if I do so.  I get the e-mails from my e-mail alert action, so I know the alert is triggering.  I replaced the rtnforwarder command line in the execute action with a simple command file to echo something to a text file, just to see if it triggered, and that didn't do anything either (although once again, the e-mail action did send me an e-mail).

It seems that "Execute an external program" doesn't work.

Does anyone have a hint for me on this?

Chris

  • Try removing the email alert from your Trap Viewer rule. So in trap viewer, you just have the "excute and external program" only.

    orion5.jpg

    I've you'd like an email, turn this option on in NCM Settings (see image below), then in the Real-Time Change Detection settings tick the "Include Real-Time Notification details in email and fill out the Email Notification Defaults. And in the Email Server Details, add in your SMTP server details.

    This is what I've done and I get an email alert each time a change is made and the running config also gets backed up to NCM. I'm not monitoring the Startup config.

    orion4.jpg

  • FormerMember
    0 FormerMember in reply to superfly

    Well, I had a support case on this, and what it comes down to is Orion broke it.

    We have seperate NPM, NCM, and additional poller servers, and until now our network people that have been simply using whichever Orion server IP they felt like when setting up the SNMP receiver.  All the Orion server IP's worked.

    Problem is that Orion started integrating NPM and NCM, and in our installation those are on different servers.  If the alert comes in on the NPM server, they have NO intelligence built in to route the requested external program call over to the NCM server, and the program only exists in the NCM installation.

    We have a few options.  We can get a complete inventory of our network, and push out changes to thousands of devices (of a wide variety of brands/models) to aim EVERYTHING at our NCM server, then undo ALL of that when Orion's merging of the two products forces us to collapse it into one server later...

    Or I can try to find a way to use schtasks.exe to schedule this job to run immediately on the NCM server so that the alert can run the executable no matter what server the trap comes in on.

    Either way, it's broken, and I have to find a workaround, and the workaround is going to be a lot of work.

  • Bugger. I have NCM & NPM on the same server so it all works fine.

    Hope you can sort it.

  • Are you aiming to keep NCM and NPM completely separate or is it ok to have NPM/NCM integrated but still on separate servers?

    If the latter, then there is a section in the NCM Administration Guide about installing NCM on a separate server and integrating into NPM.

    "Migrating NCM to a Different Server/Deployment Type - Case 3: Upgrade Non-Integrated NCM and Integrate with an Orion Product on another server" (pg 51) - http://www.solarwinds.com/documentation/orionNCM/docs/orionNCMAdministratorGuide.pdf

    1. Backup the databases just in case...

    2. Run the NCM 7 installer on the Orion Core server, choose the option "No, my primary NCM is on a different server", and go through the configuration wizard.

    3. Run the NCM 7 installer on the NCM server and choose the option "Yes I want to install/upgrade NCM and integrate with Orion on a different server".

    4. Configure the trap alerting for RTCD on the Orion core server as it should now have the NCM executable available.

    Read through the documentation and admin guide (especially the sections on integration).

    Regarding the trap configuration you can use NCM policies to detect the devices configured to send traps to the other pollers, then push out scripts to correct them.

  • FormerMember
    0 FormerMember in reply to shuth

    We have an integrated seperate server setup.

    I'm not positive about what's happening, but at the end of the support case I was being told that when a device that sent traps to any server but the NCM server ran the alert, there wasn't any way to make sure the external program was executed on a specific server.  Orion has been integrating NPM and NCM more and more closely, and they apparently no longer have seperate trap alerts.  Without seperate alerts, the actions could take place anywhere.

    An attempt was made to forward the traps to my NCM server, but that resulted in a loop, I received about 100 alert e-mails in about a minute, even though he had put on the "Discard this trap" action at the end of the list.  He told me there was no way to forward a trap from one SW server to another without looping, since in this version the trap viewing and alert system is central.  No matter which server receives the trap, it's still going to the same alert service.

    Since I hung up on my second hours long support session, I thought that I could get this to work by getting psexec from sysinternals to execute the solarwinds.ncm.rtnforwarder executable remotely from whichever server wanted to do it.

    So, if I put psexec.exe in the c:\sysinternals\ directory on each of my SolarWinds servers, the the following should work no matter which server fires it off:

    c:\sysinternals\psexec \\myNCMserver -accepteula -s "c:\Program Files\SolarWinds\Orion\SolarWinds.NCM.RTNForwarder.exe"

    This does indeed work when pasted into any command line on any of my SolarWinds servers, but when fired off from an "execute an external program" task, it doesn't run.

    In fact, I even  tried running "echo hello > c:\sysinternals\tmplog.txt"  It would run from any command line, but SolarWinds would not run it.

    You know, I've been through the first tier level, and now the engineer level, and I have made NO progress.  These people are going to get me fired.  My boss watches all happy smiling people on their promotional videos, that go on and on about how easy it is, and show 3-step setups in their demo environments, then he wonders why I don't have it set up yet.

    I mean, it only took the guy in the video 2 minutes.