12 Replies Latest reply on Dec 9, 2019 3:38 PM by fasteddye

    Alerts that execute perl

    fasteddye

      We have recently moved our NPM over to new Servers and updated to NPM 12.5.  We use NPM primarily to monitor network infrastructure (routers, switches, firewalls, APs, etc.) and have it working with our HPSM ticketing system to generate tickets on alerts.

      We have a few alerts setup that executes a program using perl when a WAN interface goes down.  The perl script telnets to a router and pulls some show run commands and then emails our ticketing system in xml format.  These alerts are triggering since we moved to new Servers and updated 12.5 however there seems to be an issue with the perl script.

       

      I have installed ActivePerl on the new Servers and have copied over all the appropriate files for the scripts from our old NPM server.  I am having trouble figuring out where I can look on the NPM server to see if the perl (.pl) file is even being exexcuted.

       

      I am reaching out to the community to see if anyone that has used ActivePerl with alerts could help point me where to start troubleshooting or logs to look at as to why the script may not be working or even being executed.  I can run the script from a command prompt on the server and it seems to run.

       

      Many Thanks.

        • Re: Alerts that execute perl
          gkjono

          You could enable Windows auditing on the Perl exe and the script to see if they are getting accessed by the application.

           

          Have you tried executing the action simulation from within the alert's trigger actions?

           

          My first guess would be that either a path is different and needs to be updated in the alert's trigger action, or at the top of each of the Perl scripts.

           

          -gkjono

          Loop1 Systems: SolarWinds Training and Professional Services

          • Re: Alerts that execute perl
            fasteddye

            Thanks so much for the reply.

             

            How would I turn on the windows auditing on Perl exe?

             

            I have run the simulation and it comes back with action executed successfully.

             

            Many Thanks.

              • Re: Alerts that execute perl
                gkjono

                This link has a quick walk through on enabling file auditing. You'd want to enable it for success and failures on the actual Perl exe file.
                Success Center

                When you simulate, does the script provide the intended results, aside from the success message? Does it pull down the configs, etc?

                 

                -gkjono

                Loop1 Systems: SolarWinds Training and Professional Services

                  • Re: Alerts that execute perl
                    fasteddye

                    I will review the link about enabling file auditing, thanks for providing.

                     

                    The simulation does not provide the intended results.  I did get an Event Message after running the simulation (screen shot below).  Not sure if this provides any value but thought I would show. 

                     

                     

                    Thanks.

                      • Re: Alerts that execute perl
                        fasteddye

                        I setup the windows audit and when I run a simulation I see the perl.exe file being accessed in the Windows Security Logs.  Now it seems I must discover what is failing or not working with the .pl file.  I am new to perl and ActivePerl so I am not sure how to see why the script is not completing or what in the script is failing.  Would there be any log files associated to ActivePerl?  I have tried to duplicate everything from the old server on the new server like Environment Variables, script paths, etc.

                         

                        Thanks.

                          • Re: Alerts that execute perl
                            mesverrum

                            Not a solution to your problem but I've had similar problems often enough that whenever I write scripts these days I always include a chunk of boilerplate code to make sure it writes a log output somewhere.   So many hours wasted trying to reverse engineer scripts that I find laying around on client systems.

                              • Re: Alerts that execute perl
                                fasteddye

                                These are scripts that have been inherited and the author is no longer with our company.  All the scripts have been working for years but we just moved over to new Servers (windows 2012 to 2016) and updated NPM (12.1 to 12.5) these scripts are not working.  I think I have been able to determine that the script is being called as it should (by doing the windows auditing) so it would be great to somehow have some log output to help see what is going on in the script is called.  I do not have any real perl experience so I am not sure how to include boilerplate code.

                                 

                                Many Thanks.

                                  • Re: Alerts that execute perl
                                    mesverrum

                                    My comment was a more general quip to everyone out there building scripts that someone will eventually have to troubleshoot.

                                     

                                    For some rudimentary debugging you can just look here for the syntax on writing the outputs to a text file and that way after the file gets called you can read the logs and see what it got.

                                    Writing to files with Perl

                                     

                                    I'd do something like roll through the code and just have it print out the contents of every variable and the results of every action as it goes until you see where it's breaking.

                                    • Re: Alerts that execute perl
                                      gkjono

                                      I'd second mesverrum. You are basically into debugging your script now. Perl is really modular, so it is possible that you don't have all of the same Perl modules installed on the new server that you had on the old. Perl could be running an different version and some of the commands in the script could be deprecated as well.

                                        • Re: Alerts that execute perl
                                          fasteddye

                                          Yes it looks like it may be Perl module related.  The old server had ActivePerl 5.1.42 and now it looks like 5.26 or 5.28 are available.  I am having trouble getting some of the modules needed installed for both 5.26 and 5.28.  The following are the modules being called.  I have been able to get Net::Telnet:Cisco and MIME::Lite installed but having trouble getting Win32::Env.  I have tried using cpan and Win32::Env fails with dependencies of Win32::Console and Win32::TieRegistry.  I have tried to cpan install both those modules but does not seem to complete.  I am also having trouble getting ppm installed and I have not had any luck building a configuration on the ActiveState fork custom build.  I keep getting an unable to build the distributions for this project when I select the Win32 modules to be included in the build.

                                           

                                          use warnings;

                                          use strict;

                                          use Net::Telnet::Cisco;

                                          use DBI;

                                          use MIME::Lite;

                                          use Win32::Env;

                                          use IO::Handle;

                                           

                                          Any ideas or pointers on how to get these modules installed or what ActivePerl version would be recommended?

                                           

                                          Thanks.

                                            • Re: Alerts that execute perl
                                              fasteddye

                                              I think I have made some progress.  I have been able to get the perl modules needed installed.  I can also test the .pl file and it sends me an email without the filled in blanks from db.  I can now simulate the alert and see it accessing the perl.exe file with windows auditing but do not receive the email.  I can test the .pl file like it was being called from NPM and receive an error which looks to be db conversion related. I did have to change the ODBC driver from "SQL Server" to "ODBC Driver 17 for SQL Server" so I am not sure if that means some other code inside the script will need to change?  The new DB servers are much stricter on TLS so we cannot use the old SQL Server drivers.

                                               

                                              This is the alert execution

                                              Execute program : perl "D:\Custom_Scripts\WANInterfaceDown.pl" ${InterfaceID} NPM-${ObjectID}-${N=Alerting;M=AlertID}-${SQL:SELECT DATEDIFF(ss,'19700101','${N=Alerting;M=AlertTriggerTime;F=OriginalValue}')}

                                               

                                              The below shows the results if I test this directly with perl by using parameters that are actually being sent from NPM to perl on this alert.  To test and generate the string I have an interface that is down.

                                               

                                              C:>perl -w d:\custom_scripts\waninterfacedown.pl "15509 NPM-15509-27-1575867875"

                                              DBD::ODBC::st execute failed: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Conversion failed when converting the varchar value '15509 NPM-15509-27-1575867875' to data type int. (SQL-22018) at d:\custom_scripts\waninterfacedown.pl line 60.

                                              Can't return outside a subroutine at d:\custom_scripts\waninterfacedown.pl line 62.

                                               

                                              I feel I am close to getting this script to work.

                                               

                                              Thanks.

                                                • Re: Alerts that execute perl
                                                  fasteddye

                                                  I have successfully been able to get the scripts to run and complete.  These scripts are using Net::Telnet:Cisco to connect to Cisco Routers.  I was feeling good about getting the scripts to complete and now wanted to look at moving over to SSH connection instead of telnet.  This appears to be harder that it seems.  I am not having any luck getting Net:SSH2 or Net::SSH2::Cisco  or any Net:SSH module to install. 

                                                   

                                                  Any experiences loading SSH module and getting it to work with Cisco Routers?

                                                   

                                                  This is the current code being used by Net::Telnet::Cisco

                                                   

                                                  #Create the connection to the node

                                                  $session = Net::Telnet::Cisco->new(Host => $nodeip, Input_log => $logfile, ErrMode => 'die');

                                                  if ($session->login(Name => $nasuser, Password => $naspass)) {

                                                      $session->enable($enablepass);

                                                      @command1 = "Interface Commands Output";

                                                      push(@command1,("\n\n$caption>sh int $ifname\n"));

                                                      push(@command1,$session->cmd("sh int $ifname"));

                                                      $session->close;

                                                   

                                                   

                                                  Many Thanks.