6 Replies Latest reply on Dec 1, 2017 10:15 AM by rschroeder

    SCP/TFTP Server Settings Confusion

    xtraspecialj

      The documentation for both the "TFTP Server Settings" and "SCP Server Settings" web pages (found by going to Settings > NCM Settings in your Orion Web Console, provided you own NCM of course) is quite sparse.  It isn't clear what we are configuring exactly.

       

      When I go to the SCP Server Settings page (seen in the screenshot below) it lists all of my polling engines and has a Username and Password field for each one, along with a place to put a SCP Server IP and the Config Transfer Directory.  I don't understand why there are separate entries for each polling engine.  What do the polling engines have to do with setting up the SCP Server Settings?  Is it intended that we setup a SCP server on each polling engine?  If so, why?  That sounds like a pain in the **** as well as completely unnecessary. 

       

      What we've done is setup a SCP server on our NTA Database server.  So our SCP server doesn't live on any of our polling engines. 

       

      Since we are just using one SCP server, how should I setup the screen below?  Do I just put in the same Username/Password combo along with the same SCP Server address and Directory under each polling engine's section on here?  If so, that is what I tried, however I don't know what to put in the config transfer directory box.  If I put anything other than "/" in it, it won't validate, even though I can successfully transfer stuff to/from the directory via SCP and I've validated that the Windows permissions to that folder allow Everyone to access it (for now while I'm testing it.  I'll tighten that up once I get it working).  The Root directory on our SCP server is "E:\SFTP_Root".  If I put that in on the TFTP Server Settings page, that works, but if I put it in here, it doesn't.  Only "/" works.  I'm so confused why this doesn't work and it just isn't clear what I'm supposed to do...

        • Re: SCP/TFTP Server Settings Confusion
          rschroeder

          When you configure the TFTP and SCP pages, you're telling your pollers where to store files, or where to retrieve files from.

           

          You don't have to setup separate directories on every Solarwinds server manually--it's done automatically for you when you install the product on each server.  The config page allows you to tweak & fine tune & customize where the files are stored, and what credentials are used for that access.

           

           

          Solarwinds NCM is built so that the main instance of NPM/NCM and every Additional Polling Engine (APE) has its own local Config Transfer Directory.  This enables you to get a lot more work done in any given time.  For example, I have five Polling Engines.  Each one manages about 20% of my nodes, volumes, and interfaces (up to 100,000 of each!).

           

          Now suppose you have a Running-Config Backup job that runs every day--maybe multiple times each day.  AND you have a Startup-Config backup job that also runs daily.

           

          Then suppose you have a thousand nodes that this single job runs against.

           

          By having five APE's, each with their own SCP server and directory, the job takes 1/5 as long to complete--or less--than it would if you had just one poller.  With one poller managing all your nodes' config files, it might take so long to complete the job that it would run into the next scheduled instance of that job.

           

          Using a separate directory per polling engine means, you don't have all of your files in the same location (i.e.: Not all of your eggs are in one basket).

           

          This is neither good nor bad; just that this appears to be how Solarwinds is built by default.

           

          You could create a Solarwinds Service Account in AD and use the same name and password for all your pollers.  Or for enhanced security, select a different username and password for every poller--it's your choice, SW has given you some flexibility here.

           

          They've also given you the option to specify a config transfer directory--one unique one per polling engine.  You don't have to use it.  You can certainly send the configs all to the same location.

           

          AND you can manually set the option to use a third party SCP server if you don't like Solarwinds for any reason.

           

           

           

          As long as the credentials set to access the directory are correct, and the directory is present in the path provided (and can be written to), AND you can successfully validate each directory's settings for path & credentials, you should be good.

           

          If you can't validate the directory or credentials, fix that.  It's certainly handy to have the same path & credential name on every server.  And it makes backing up those servers simpler and faster than having all the files in one transfer directory.

           

          Solarwinds has put you charge, given you the ability to customize NCM's configuration to whatever makes the best sense for your environment.  You don't have to do it their way.  Set it up so it works best for you & your IT teams, and meets your convenience and resource and security requirements.

           

          And share your success with us in a response, please?

           

          Swift Packets!

           

          Rick Schroeder

            • Re: SCP/TFTP Server Settings Confusion
              xtraspecialj

              Thanks for the reply rschroeder.  So what you are saying is on the SCP Server Settings page I'm not actually telling NCM how to communicate with the SCP server, but instead I'm telling it what directory on each polling engine to use along with the credentials needed to access that directory?  I definitely don't want to setup a SCP server on each polling engine as we have 8 polling engines.  I'd far prefer a single SCP server that each polling engine sends the data to for the device/devices that it backs up.  We will have less than 20 devices in total that will be using SCP to transfer configs, so there really just isn't a need for all that. 

               

              I definitely understand the concept of splitting up the workload across polling engines for NCM and we do that for all of our device backups already since they use SSH and our devices are evenly split across our 8 polling engines.  Typically we use SSH for everything, but we are trying to get F5 binary configurations backed up and Palo Alto configs backed up.  F5's will use SCP along with the Binary config settings (which is a whole other nightmare of confusion in regards to the lack of explanation that SW gives in its documentation) and Palo Alto's work best when using SCP/TFTP to transfer config files instead of SSH, so that's why we're finally trying to get this setup.

               

              I apologize if I'm being dense, I still just don't understand how to set this page up for a single SCP server.  The vague nature of the terminology on this page and their documentation leaves a lot of room for interpretation in regards to what kind of data they are asking for here.  I'm sure once I understand it then it'll make sense, but the point of documentation should be to make it very clear to those who don't understand it.  I've been working with SW Orion products for over 6 years now and I have no clue what to do on this page...

              • Re: SCP/TFTP Server Settings Confusion
                xtraspecialj

                So I've figured out a little more, but I am still confused as to why it works this way and if there is a better way. 

                 

                Do you mind validating if this is correct and if there is a way to do it better?

                 

                - The credentials should be the SCP server credentials so that when a Device Template has the variable ${SCPServerPassword} it would resolve to this.

                - The SCP server IP address is obviously the address of the SCP server, whether that be the polling engine itself, or a standalone SCP server on a different machine.

                - (This is where I don't understand the setup) - The config transfer directory is a local folder on the polling engine where it should store the configs that it gets from the SCP server.

                 

                If I am correct with my third bullet point above, what I don't understand is what is the process NCM uses in the background when a config is backed up using a single third party SCP server like I'm doing?  A polling engine kicks off the backup on a device by sending the SCP transfer command to the device, specifying the SCP Username, Password, and ${StorageFileName} (Not sure what that would resolve to either to be honest...).  So now the config is transferred to the SCP server.  Since I'm having to specify a Config Transfer Directory that lives on the local polling engine though, how does NCM get the file from the SCP server to store it on the local polling engine?  Should I just put the network address of the SCP server directory in each location so that it stores all of them there?

                  • Re: SCP/TFTP Server Settings Confusion
                    rschroeder

                    Each of your polling engines takes on the task of being the local NCM delegate for contacting nodes and downloading configs from them.

                     

                    That means each of your polling engines already is an SCP server (or whatever method/protocol you selected for downloading configs from nods).

                     

                    There is no need to set up each polling engine with a unique destination folder for storing config files, but it's easy to do so.

                     

                    Although each polling engine does have an area reserved for this, it also has two other options:

                     

                    1. In the space provided for the destination directory to which configs will be stored (per polling engine), you CAN enter a proper network path to a shared directory, one with rights provided for your NCM service account.  It can be on your NCM server, or on anything in your network.  It just has to be a proper path, correctly configured share for the directory, and it has to be able validate the credentials of NCM as NCM tries to move config files to that server.  And every one of your polling engines can use this same destination path & directory, resulting in all your config files being stored in the same directory.  As long as you've got the option enable to ensure all have unique names, that's a fair solution.  Or . . .

                     

                    2. You can check the box at the top of the page to use a 3rd party server.  The regular provisions for valid network path and proper credentials apply.

                     

                    Beyond this, contact Solarwinds Support with some specific questions and requests for recommendations about the setup that's right for your information.

                      • Re: SCP/TFTP Server Settings Confusion
                        xtraspecialj

                        Yep, I ended up just putting the same SCP Server Username, Password, and IP address in the box for all of the Polling Engines so that they all pointed to my single SCP server.  Then, I shared the Root folder on the SCP server and entered the UNC network path to that shared folder in for the config directory on all of the devices. 

                         

                        To me it just makes way more sense to have a single SCP server and a single directory where it stores all of the files.  Maybe if all of our network devices used SCP/SFTP to download/upload configs, I could see the advantage of splitting the workload up between all of the polling engines, but to me that just seems like way too many directories to have to manage and browse to when you wanted to access certain configs. 

                         

                        We did the same thing for our Config Archive directories a while back because having a separate folder on each of our 8 individual polling engines (plus we are soon adding two more polling engines in our remote data center) that contained the text config files was just a pain in the rear when somebody wanted to access one.  You had to find out which polling engine the config was on and then RDP to that polling engine (or access it via the admin share) to get it.  Not a huge deal for me, a SolarWinds Administrator, but it was definitely a pain for our Network guys who don't use Orion day in and day out.  It just made way more sense to have everything saved to a centralized directory that they could access whenever they want.

                         

                        Thanks again for the help rschroeder.  I do understand it now, although I don't necessarily agree with their methodology and I definitely don't agree with the very vague documentation they provide for these settings.  It wouldn't take much to both label that settings page in a more clear and concise manner and to update the Support article and Admin guide to make it much more clear what these settings do and why.

                          • Re: SCP/TFTP Server Settings Confusion
                            rschroeder

                            It sounds like you have the makings of some good product improvement suggestions.  Take a couple of minutes, Click on Create in the upper right quadrant of Thwack, and pull down to select the right area to create a new Idea or feature. 

                             

                            Be as detailed and thoughtful and creative as possible, to help ensure Solarwinds developers get the complete picture of what you have in mind.

                             

                            Once created and submitted, remember to vote it Up, so you can get a trend started.