19 Replies Latest reply on Jun 12, 2015 2:26 PM by chandran_a

    Need best architecture for implementation

    chandran_tcs

      Hi Experts,

       

      we need your assistance in designing the best solution.

       

      we have bought the following products and allocated 4 servers for implementing all these component in failover mode.

      NPM, NCM, NTA, VNQM, IPAM, Failover Engine

       

      so please suggest how can we install?

       

      Thanks in advance.

       

      Reagrds,

      Chandran

        • Re: Need best architecture for implementation
          silverbacksays

          Do you have more than one poller? If not then:

           

          1 x SQL server

          1 x Primary Poller (all the modules)

          1 x NetFlow Database Server

          1 x FOE Secondary Server (clone of Primary Poller P2P/V'd to the fourth server)

           

          Basic steps (I'm assuming the OS is installed on all boxes already):

           

          1. Install SQL the database server & create the instance to house the Orion Database, repeat this process on the NetFlow database server (could use a MySQL or SQL Express if you're tight on SQL licenses for this)
          2. Install all SolarWinds products on the Primary Poller (and the the NetFlow database on the NetFlow server)
          3. Clone your primary server (easy enough if it's a VM, use Ghost or something similar if a physical).
          4. Install FOE on the primary
          5. Install FOE on the seconday
          6. Run FOE on the primary and confirm it has synced with it's clone.

           

          More detailed info for FOE can be found in this document:

           

          http://www.solarwinds.com/documentation/failoverengine/docs/Failoverinstallwalkthrough.pdf

           

          Hope that helps.

          1 of 1 people found this helpful
            • Re: Need best architecture for implementation
              cmgurley

              Good high-level breakdown. I see that the cloning recommendation is straight from the PDF, but have you run this in production? SQL Server has never been fond of host name changes and AD, of course, would require a sysprep to keep things savvy with the SID. MS TechNet talks about cloning SQL servers more here: Installing SQL Server in a Virtual World - Insufficient data from Andrew Fryer - Site Home - TechNet Blogs

               

              It's always seems to hang up the great clone/template features in the virtual world (aside from simpler, sysprep-able server instances). If I'm outdated with this info, please enlighten me . It'd make my day!

              • Re: Need best architecture for implementation
                chandran_tcs

                Thanks sir.

                 

                we have all four VM's only not the physical box.

                 

                Earlier we have the plan of implementing as below.

                 

                Server 1: SQL Server

                Server 2: Primary NPM, NTA & VNQM (Flow storage DB includes in this server)

                Server 3: Primary NCM & IPAM

                Server 4: Secondary for all components (NPM, NCM, NTA, IPAM, VNQM & Flow storage DB)

                is it feasible? will it be good in performance wise?

                 

                please help us in understanding the below points:

                 

                can we use same SQL server for both primary and secondary solarwinds products like sever 2 primary NPM and server 4 secondary NPM?

                do we need to create separate DB for each primary and secondary NPM?

                do we need separate server for NTA DB? that much space will it consume?

                having all the modules (NPM, NTA, NCM, IPAM, VNQM) in one server will make any issue like memory, performance etc?

                in which order we can install all these components? 1st NPM 2nd NCM like wise

                 

                Your inputs will be very much helpfull for us.

                 

                Thanks in advance.

                  • Re: Need best architecture for implementation
                    silverbacksays

                    When using SolarWinds Fail Over Engine, you need to have all the modules on the primary server, and the same on the secondary, so if you're using FOE, your planned implementation will not work I'm afraid.

                     

                    To your questions:

                     

                    1. It's not recommended to use the same database server for both NTA and the main Orion database. While there's nothing stopping you from using the same SQL server, it defeats the object of splitting the work between two database engines / servers.
                    2. No, you use the same database for both. Think of the secondary as a passive version of the primary poller, since it'll only ever be used if the primary poller goes down (I'm again assuming you're thinking of use FOE in your example)
                    3. It is recommended that you use a separate server to house the NTA NetFlow database.
                    4. As long as you deploy the modules on hardware/virtualised hardware which meet the requirements in the installation guide, you should be fine.
                    5. Primary first, starting with NPM. Whichever order you do the others is up to you, but you'll need to have the second database server up and available before you install NTA.

                     

                    Hope that helps. I recommend you read the FOE implementation document thoroughly before you start

                    1 of 1 people found this helpful
                    • Re: Need best architecture for implementation
                      RichardLetts

                      This will not work, and I feel this is something your reseller should help you with.

                      Whoever sold you solarwinds should be able to help you with these questions & help with the architecture.

                      • Re: Need best architecture for implementation
                        Craig Norborg

                        Hmm...  Let's go over your questions:

                         

                        can we use same SQL server for both primary and secondary solarwinds products like sever 2 primary NPM and server 4 secondary NPM?

                         

                        One thing you have to understand is a huge benefit to Solarwinds is the integration of the products.  Due to this, you really want to install all your modules on the same server.  This is true for MOST of solarwinds products, including all of yours I believe.  There are some exceptions like the firewall product that want to be on their own server, but will still integrate to some extent.  But you will lose features if you want to install these products on different servers.  The same is true for their SQL server, the SQL server is key to getting good performance out of your Solarwinds environment.  The SQL server NEEDS to be a beefy server to handle everything, you might get away with putting it on a VM, but it might need to be its own server.  With newer versions of NTA, they stopped using SQL and instead are using a NoSQL server, so you want this on a separate server also.

                         

                        do we need to create separate DB for each primary and secondary NPM?

                        No, the failover-engine doesn't work that way.

                         

                        do we need separate server for NTA DB? that much space will it consume?

                        Yes!!  It's not an SQL server database anymore.    It has beefy requirements and wants local storage.    Look here for more info...  SolarWinds Knowledge Base :: NTA 4.0 Installation: Frequently Asked Questions

                        The amount of space depends on the # of nodes and interfaces you monitor.  We are monitoring a ton of them and our DB is using 91Gb right now...

                         

                        having all the modules (NPM, NTA, NCM, IPAM, VNQM) in one server will make any issue like memory, performance etc?

                        Probably not if you size the server correctly.    You really want a separate SQL database server and NTA (NoSQL) database server for performance.

                         

                        in which order we can install all these components? 1st NPM 2nd NCM like wise

                        NPM first for sure, the rest is probably a bit flexible, but I'd do NPM, NCM, NTA, IPAM VNQM.

                         

                        silverbacksays was spot on in his server recommendations, although he might have missed you're planning on doing all this on VM's.  You can give that a try, but might want to move the SQL and NoSQL (NTA) databases to real physical boxes.

                         

                        For future potential performance boosts, you can look at additional polling engines to offload the primary box - NPM will guide you as to when you need this, probably not right away even if you're on a VM.  Another possible speed boost for users is to put the web server on a different box via the AWS (Additional Web Server), which can offload your main polling engine a bit (for very little money)...

                        1 of 1 people found this helpful
                          • Re: Need best architecture for implementation
                            cmgurley

                            Craig,

                             

                            I'm curious about your repeated push for physical servers in the deployment. Can you identify what limitation of modern hypervisors precludes the use of a virtual machine for even hefty services like SQL or NoSQL in the above deployment? Granted, we don't know Chandran's environment, so we can't speak to the socket, core and RAM count of his physical hosts, but that point works the same way about spare hardware he might have.

                             

                            You are fully entitled to a different viewpoint, but unless his needs demand a CPU count in the double-digits, VMs are not just tolerable, but actually preferable to a physical deployment. Thanks.

                              • Re: Need best architecture for implementation
                                Craig Norborg

                                I never said he had to use physical servers, it may be perfectly fine in his environment to use VM's, but he gave us absolutely no insight to his environment.   He seemed really concerned about performance though, so I'm assuming its probably a fairly large environment.  If his 4 VM's that he's speaking about are all on the same physical server and its already 80% utilized with a number of other VM's, its probably not a good idea to put his SQL server or NoSQL server on it.   If the storage attached to the VM system can't equal the requirements needed by the NTA flow storage server (in his environment), then he'd be better off moving it to a physical server.   I know we quickly moved away from a VM for our environment, but we're processing quite a bit of netflow traffic.    I know our SQL guys were much more in favor of them putting our SQL databases on one of their physical servers that they could share with other systems over making an "Orion-only" VM with SQL.

                                 

                                Don't get me wrong, there is absolutely nothing wrong with having VM's for any of your servers, but it highly depends on your environment and how much data you end up pushing.   He gave us no insight as to what exactly he's doing.   There does come a point in time where if you're VM is consuming enough resources it makes sense to make it it's own physical server, or at least the only VM on a host.

                                 

                                I was trying to stress more the need for loading all of your different components (NPM, NTA, NCM, IPAM, VNQM) all on the same server for getting 100% integration of the components.   And that silverbacksays was correct in his assessment of how to utilize the 4different servers they were getting.   ie: one for SQL, one for all the modules, one for NoSQL and then one for FOE.   From how he was asking, it sounded like he was reading some older recommendations back when NCM or IPAM weren't as fully integrated as they are now.  You no longer want them on separate servers.

                                  • Re: Need best architecture for implementation
                                    cmgurley

                                    Sounds good and agree, Craig. I incorrectly gained the impression of a negative bias against VMs. In our environment we use hosts with four sockets and 10 cores per socket so they can handle beefy VMs. On top of that, backend shared storage more than provides for the VMs' I/O needs.

                                     

                                    Makes sense with the service consolidation for integration benefits. Definitely spec the solution accordingly to fit the need. We all just need more info from Chandran to advise intelligently.

                                     

                                    Thanks for indulging my questions. A few folks in the larger community still think of VMs as second class citizens. I'm glad you're not one of them. Cheers.

                                • Re: Need best architecture for implementation
                                  chandran_tcs

                                  Thanks All.

                                   

                                  Each VM's are in different servers and below are the size of it.

                                  you mean we can use a single DB for all the modules (NPM, NCM, IPAM, VNQM) as we are installing all together in a single server.

                                  or a separate DB for each and every one?


                                  HDD           Ram           DataCentre

                                  1 TB           16 Gb          DC1

                                  1 TB           16 Gb          DC2

                                  500 Gb        12 Gb          DC1

                                  500 Gb        8 Gb            DC1        

                                   

                                  We will go with the one for SQL, one for all the modules primary, one for NoSQL and then one for FOE.

                                   

                                  please suggest which VM to be allocated to Primary Orion, SQl server and NoSQL.

                                   

                                  I believe the Flow storage DB comes with the NTA binary please confirm?

                                   

                                  Thanks in advance.

                                    • Re: Need best architecture for implementation
                                      Craig Norborg

                                      Ok, so more questions before I answer again.   How far apart are DC1 and DC2, and what kind of latency do you have between them?    What kind of processing power does each box have?   How is the storage on each box laid out?

                                       

                                      I can give a bit of guidance based on this though.   Yes, you want to have a single DB for all the modules (not including the separate Flow Storage DB).

                                       

                                      I tend to go for the beefiest machine being my SQL server, with the second beefiest my main poller.   Some might disagree and want your FOE to be the exact same as your primary poller though.  Personally my train of thought on this is the FOE is only applicable when the main server dies, so its going to be sitting around most of the time doing nothing.   I'd personally put it on the smallest machine capable of handling things, which any one of your servers is capable of handing it I think.   So, my initial thoughts would be in order SQL on box 1, NPM and modules on box 2, NTA flow storage on box 3 and FOE on box 4.

                                       

                                      However, the second box being in a different DC will probably change this assuming they're some geographic distance apart and not simply in a different building on the same campus.   You really need to have your main box and SQL server in the same DC, not to mention the flow storage DB should be too.   Which would mean SQL on box 1, NPM and modules on box 3, FSDB on box 4 and FOE on box 2.   That as a given, I would not want to run the FOE for long with any significant latency with it's databases in a different DC.   Once again, this is assuming there is some distance/latency between the two DC's...

                                       

                                      Not sure what your question is about the Flow Storage DB coming with the NTA binary.   NTA comes with everything you need to get up and running.  When you follow the recommendations and put the FSDB on a different server, your main poller is still the one to receive and process all the Netflow packets, it just uses this separate server for storing the data.    So think of the FSDB server as just a second database server, nothing else.   It doesn't receive Netflow packets or serve up NTA web pages..  I believe when you install NTA it directs you to first install the FSDB on the server, then install the package on the main poller and it will connect to the FSDB at that time and verify that everything is ok.

                                        • Re: Need best architecture for implementation
                                          RichardLetts

                                          box 4 is not really large enough for the FSDB/NTA flow server: NTA Flow Storage Database Requirements

                                          Needs 16GB of RAM according to the docs.

                                          • Re: Need best architecture for implementation
                                            chandran_tcs

                                            Thanks Craig.

                                             

                                            yes the DC's are present in different geographic location. sorry am not aware of other things like processing power, storage etc.

                                            Regarding FSDB for NTA, do we need to buy a separate SQL license or it is inbuilt DB in NTA?

                                             

                                            Any recomendation in sizing? please let me know i will insist my server built team to do so.

                                             

                                            Thanks in Advance.

                                              • Re: Need best architecture for implementation
                                                silverbacksays

                                                Hi chandran_tcs!

                                                 

                                                It'll need a separate database instance, but as with all SolarWinds products you could just use SQL Express rather than going for a new full database license. Weigh up performance/scalability/reliability requirements in order to make your decision there

                                                 

                                                With regards to sizing, it's impossible to make a real suggestion as the size of it depends on how many devices you'll be handling flows for. Take a look at the following link for advice on Orion database management:

                                                 

                                                http://www.solarwinds.com/documentation/Orion/docs/OriondbBestPractices.pdf

                                                • Re: Need best architecture for implementation
                                                  Craig Norborg

                                                  RichardLetts is correct, the FSDB specs do say 16Gb of RAM, so any box you put that on should have that amount of RAM.   The FSDB doesn't need a SQL license at all (especially since it's not SQL!), it includes everything you need for running the FSDB for NTA...

                                                   

                                                  silverbacksays gave a good link for sizing recommendations.   The main recommendation I would give is that all normally active components (NPM and modules, NTA FSDB, SQL Server) should all be in the same DC, if you have to have one component in a different datacenter, I'd make it the FOE.   That being said, I wouldn't want to run off the FOE for long periods of time.   That is unless you have a backup of the SQL server at that remote location also, where you keep a copy of your data and can run from there...    While I believe they used to say you could run polling engines and such from geographically diverse locations, I believe now they recommend that all components are in the same location.   This is not only because there is quite a bit of data flying between the polling engine(s) and the database server(s), but also because if you poll the same node from different locations in your network, you will get varied statistics for latency and such.   This can throw off reports and alerts that might be based on those figures.

                                                   

                                                  I think with some tweaks based on the specs and best practices docs, your hardware will probably be fine.   Definitely to get started, you might need to tweak it as you go on also.

                                                   

                                                  The main thing I'd say is install it and start having fun!!   Esp. since they're VM's you should be able to fairly easily do things like increase memory or processing power to the boxes if they should need it.    At worst you chalk it up to a learning experience and start over, at the very least you could probably restore your database so you aren't even starting from scratch at that!

                                                   

                                                  Go!  Have fun!!

                                              • Re: Need best architecture for implementation
                                                RichardLetts

                                                I think that you will be unhappy with this install on that hardware; I foresee that will be back here complaining the performance of your installation is bad, that our advice was bad, and that Solarwinds/Orion sucks.

                                                I am sorry, but as a professional I do not have enough information about your environment to design a solution for you . You have had some good advice here, but we're trying to guess about your environment.

                                                 

                                                It's time to invoke Dukhat; I feel that you have done a foolish thing and bought software without engineering the server infrastructure to support it.

                                                 

                                                You should work with solarwinds, or whoever sold you the product, to determine the right server configuration that will work for your environment.