27 Replies Latest reply on May 4, 2017 5:16 PM by supportdiehard

    Data is not available NTA issue

    davetza


      We had Solarwinds Orion with NTA, NPM, NCM installed by a Solarwinds Gold reseller in December 2013. NTA was at V4.0 at that stage and the reseller did not specify the flow collector had to be on a seperate server. Netflow has been working fine and has been a very useful tool. In the last week I have been getting an issue where if I click though to a Netflow node I get data not available for all the catergories. The NTA summary page still works and I can see nodes are receiving netflow data and I can see top talker details for the whole estate etc. This is very frustrating as we are having P1 issue with WAN utilisation at one of our sites and Netflow is crticial to keeping an eye on what is going on.

       

      THe reseller have been as helpful as they can be but have not been able to help. They also pointed out that the min spec changed for NTA in January and we now should have a seperate machine for flow data. This is not great from my end as I just speced and built the the app 2 months ago and now I'm been told I need a new server!!! I raised a call with Solarwinds but they took one look at the CPU load and said that was the issue. I have subsequently upgraded the number of CPU's on the main poller which has brought the load done significantly. I have also reduced the number of Netflow flows to 1 from around 20 but I am still having the same issue. Does anyone have any suggestions as to what it going on and how I can get this fixed. I have a request to get a new machine stood up for the flow data but this will take about 2 weeks and I need the Netflow data ASAP.

        • Re: Data is not available NTA issue
          jswan

          Can you put the flow database on a VM? It doesn't need to be a physical server. From what I've seen the CPU utilization is pretty low once you get away from writing flows to SQL Server.

          • Re: Data is not available NTA issue
            njoylif

            I'm having the same issue where data is not available.  I'd love to have SW chime in.  I'm not having resource issues in my env. though.

            • Re: Data is not available NTA issue
              kthomason

              I am having the same issue.. this started this morning. i was fine yesterday. i checked the permissions on the services and they  are all set correctly. would like help with TS'ing if anyone has seen this before..

              • Re: Data is not available NTA issue
                06s197

                I am having the same issue as well. No performance issues here, 16GB RAM and 4 virtual procs. Started happening last week.

                • Re: Data is not available NTA issue
                  terrancekennedy

                  Have you guys tried looking at the disk space on the flow database server?

                  If it's less than 5% or 10% then you may not be storing current data and therefore it will not display data.

                  Also check your data retention policy you may need to lower it.

                    • Re: Data is not available NTA issue
                      06s197

                      I have 40% free (40GB) I don’t think space is an issue, It has been the same retention settings as V 3.9, was just broken after the 4.0 upgrade. I did patch this morning to 4.0.2 and its still happening

                      • Re: Data is not available NTA issue
                        kthomason

                        i have over 100gb avail. also running 16gb ram. i have be emailing back and forth to solarwinds tech support and they suggested i run an NTA repair. which seems to have partially worked i am waiting 2 hours for data to amass to verify. i am getting my charts now with the exception of "Top 20 Receivers". I will update in a little while once i verify that the repair worked. I will also post the instructions on how to do the repair on NTA,

                        • Re: Data is not available NTA issue
                          kthomason

                          Ok folks looks like I am all good now. So here is a step by step of what i did.

                           

                           

                          Repair of NTA

                           

                           

                          1.) On the server running NTA open Control Panel

                          2.) now open Programs and Features.

                          3.) Locate "Solarwinds Newflow Traffic Analyzer"

                          4.) Right click on "Solarwinds Newflow Traffic Analyzer" and left click on repair.

                          5.) Take all defaults as they should be populated with existing NTA locations.

                          6.) Let it run all the way through.

                          7.) Click Finish if asked and Launch to your solarwinds netmon website.

                          8.) Some charts and graphs will repopulate immediately however i would give it 2 hours for everything to come back to normal.

                           

                           

                          After doing the above and still having and issue i would then suggest you to contact Solarwinds tech support.

                              • Re: Data is not available NTA issue
                                networkor


                                I also have the error, Data is not available ...

                                 

                                Running NTA 4.0.3 and DB is on a separate server, I have run every diagnostic and check that I am aware of, re-installed NTA, but it will not
                                provide any graph data.  I should note that I do see changing data stats under “Top 10 NetFlow Sources by % Utilization” and if I add or change a remote device, I see the traffic warning “NetFlow Receiver Service [ORION01] is receiving flow data from xxxxxx”

                                 

                                At this point I am totally lost on next steps, so any help would be appreciated

                              • Re: Data is not available NTA issue
                                shaochen

                                 

                                This is what I am getting for the last few days. I also have a case open to see why this is happening. It takes quite a long time for it to display. We have 3 VMs for each build, web console, NPM SQL database and NTS storage.

                                To some point, when I click on the NETFLOW tab, the page will try to load and it never loads. I can see the browser tab icon goes in circle and circle for 1 or 2 minuts, and it is still on NPM Home page after the wait.

                                Just followed what kthomason suggested, now I am at least getting NETFLOW page to show but graphs are still trying to load.

                                 

                                This is what my NTA storage VM disc info looks like if you wonder the storage size is low...

                                 

                                Any suggestions?? Thank you in advance.

                            • Re: Data is not available NTA issue
                              06s197

                              Still has not solved it for me. Works for a little while, and then goes back to its old ways. I will wait for Solarwinds to chime in, or open a ticket.

                              • Re: Data is not available NTA issue
                                06s197

                                I have opened up a case with Solarwinds, I will let you know what we find (Case # 621571)

                                • Re: Data is not available NTA issue
                                  06s197

                                  Here is the response I received after they reviewed my diagnostic log files. I have increased my drive space by 50GB, even though I have 33GB free. I will let you know if it works

                                   

                                  Tim,

                                  Looks like the last time this happened was on Friday. What I'm seeing in the logs is that the flow storage database is periodically running out of space on the drive. Here is what I am seeing:

                                  2014-05-16 09:17:23,580 [STP SmartThreadPool Thread #23] ERROR SolarWinds.Netflow.Processing.Workflow.NetFlowPacketWorkflowManager - Error flushing data
                                  System.ServiceModel.FaultException: Exception 'IOException' while appending data to FastBit: Disk is out of space

                                  The way that this version stores data in it's database is by first attempting to verify if the disk has space or not to write the data to. It does so to make sure that when it writes the flow data to the disk, that it will be able to all in one go to avoid any breaks in the write that may cause data corruption.

                                  The way it does this verification is to write shadow partitions to the disk in 25mb blocks until it reaches the size that the flow data block is in. So if the flow data that needs to be written is 10gb, it will continue to make those shadow partitions out til it takes up 10gb of space, then remove them and add the data in. This can lead to the disk growing and shrinking constantly in size. If at any point it reaches the capacity limit while doing the verification, it will continue to try until there is enough space to do so. This can cause NTA to stop collection data at that point.

                                  Currently there is 32.9gb of free space on the C:, but this could be taken up at any point. What we recommend is to move the flow storage database to another server, or to another large partition on the same server. If that is not possible, try expanding the partition of the C: out further by another 50-100gb.

                                  If you decide to move the flow storage database, here are steps on how to do so:

                                  http://knowledgebase.solarwinds.com/kb/questions/4902/Changing+the+NTA+Flow+Storage+Database+Location#LocalToAnotherDisc


                                  Please let me know if this helps or if you require further assistance.

                                  • Re: Data is not available NTA issue
                                    jeffnorton

                                    All,

                                       Solarwinds support issued a two DLL replacements for the NTA Flow server that fixes the issue.  The issue is the Solarwinds NTA flow server service on the server was hanging at the 2.5 hour mark (may be different depending on the amount of flows you collect).  I've had the patch since Tuesday and no more process hangs

                                    • Re: Data is not available NTA issue
                                      06s197

                                      It looks like SW pulled 4.0.2 from the site. Mine shows the latest as 4.0.1 now. Jeffnorton did they say if they were going to publicly release a hotfix/patch or do we have to contact support to get it?

                                      • Re: Data is not available NTA issue
                                        shaochen

                                        At this moment I have a ticket open to look into this issue. I have noticed the Top XX Conversation module is taking a long time to load and it slows down other Top XX modules on the NTA Summary page. I believe that is what SolarWinds tech support is checking to find out why.

                                        • Re: Data is not available NTA issue
                                          choly

                                          Have you already tried to contact SolarWinds Support?