cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

NTA version 4.4 is available

NETFLOW TRAFFIC ANALYZER

Faster. Leaner. More Secure.

The new NetFlow Traffic Analyzer leverages the power of columnstore technology in MS SQL Server to deliver answers to your flow analysis questions faster than ever before. MS SQL 2016 and later runs in a more efficient footprint than previous flow storage technologies, making better use of your infrastructure. Support for TLS 1.2 communication channels and monitoring of TCP and UDP Port 0 traffic helps to secure your environment.

Version 4.4 also introduces a new installation process to confirm that you have the necessary prerequisites, and to guide you through the installation and configuration process.

NTA 4.4 is now available in the Customer Portal. Check out the Release Notes for an overview of the features.

Faster

The latest release of NTA makes use of Microsoft’s latest version of their SQL columnstore based flow storage database.  Columnstore databases organized and query data by column, rather than row index. They are the optimal technology for large-scale data warehouse repositories, like massive volumes of individual flow records. Our testing and our beta customer experiences indicate that columnstore indexes support substantial performance improvements in both querying data, and in data compression efficiency.

NTA was an early adopter of columnstore technology to enhance the performance of our flow storage database. As Microsoft’s columnstore solutions have matured, we’ve chosen to adopt the MS SQL 2016 and later versions as the supported flow storage technology. That offers our customers the ability to standardize on MS SQL across the Orion platform, and to manage their monitoring data using a common set of tools with common expertise. We’ve made deployment and support simpler, more robust, and more performant.

Leaner

This same columnstore technology also runs more efficiently with the existing resource footprint. This solution builds and maintains columnstore indexes in memory, and then manages bulk record insertions with much less intensive I/O to the disk storage. CPU required to build indexes is also substantially less intensive than our previous versions. As a result, this version will make better use of the same resources to run more efficiently.

More Secure

This version of NTA supports TLS 1.2 communication channels, required in many environments to secure communications with client users.

Beginning in this version, NTA will explicitly monitor network flows that are destined to TCP or UDP service port 0. Traffic that’s addressed to TCP or UDP port 0 is either malformed – or malicious traffic. This port is reserved for internal use, and network traffic on the wire should never appear addressed to this port. By highlighting and tracking flows addressed to port 0, NTA helps network administrators to identify sources of malicious traffic that may be attacking hosts in their network, and providing the information they need to shut that traffic down.

NTA will surface port 0 traffic as a distinct application, so the information is available in all application resources.

Screen Shot 2018-04-26 at 4.43.35 PM.png

Supported Database Configurations

This version of NTA maintains a separate database for Flow Storage. NPM also maintains the Orion database for device and interface data. Both of these databases are built in MS SQL instances.

New installations of NTA and upgrades to version 4.4 and later will require an instance of MS SQL 2016 Service Pack 1 or later version for flow storage. For evaluation, the express edition is supported. For production deployments, we support the Standard and Enterprise editions.

When upgrading to this version from older version on the FastBit database, data migration is not supported. This upgrade will build out a new, empty database in the new MS SQL instance.  The existing flow data in the FastBit database will not be deleted or modified in any way. That data can be archived for regulatory requirements, and customers can run older product versions in evaluation mode to temporarily access the data.

In the current NTA product, we require a separate dedicated server for Flow Storage. The simplest upgrade would use that dedicated server with the new release to install an instance of MS SQL 2016 SP1 or later for flow storage. Many of our customers will be interested in running both the Orion database and the NTA Flow Storage database in the same MS SQL instance. We support that, but for most customers that will take some planning to consolidate and to appropriately size that instance to support both databases.

Here's a more detailed discussion of NTA's New MS SQL Based Flow Storage Database. Also, a knowledge base article on NTA 4.4 Adoption is available, with frequently asked questions.

We’re doing some testing now to provide some performance guidance for key performance indicators to monitor. One of the benefits of using MS SQL technology for both of these databases is that there are many common tools and techniques available to monitor and tune MS SQL databases. We plan to provide guidance for both monitoring, and deployment planning.

Conclusion

Please visit the NetFlow Traffic Analyzer Forum on THWACK to discuss your experiences and new feature requests for NTA.

Comments

Hi,

Has anyone migrated their NTA instance to SQL yet?  If anyone has, has the size of the database increased from previous versions?  I am upgrading NPM today but am waiting on doing NTA.

any insight into alerting based on NTA? i searched in the forum but i dont think its available yet... can anyone confirm?

Hi orionfan​, when you upgrade and things have settled a bit, would sure like to talk to you about your thoughts about the upgrade process, the new version and NTA in general.  Please reach out when it's a good time to share your feedback in a WebEx session with User Experience (UX).  Thanks!

Hi Kellie,

I'll let you know when I have this done.  I don't have a timeframe on this.  We are running into a couple of issues with NPM and UDT since the upgrade to 12.3 that I'd like to get resolved first.  Plus, I believe the SQL server that houses our NPM database is being upgraded to SQL 2016 in the near future so I may wait to do it then.

It's not available yet - it's a feature we're working on now. For what kinds of conditions would you like to be able to trigger alerts?

No migration of old flow data already in the fastbit storage? Come on!

Trying to find out what firewall ports need to be opened when upgrading from NTA 4.2.3 to 4.4

The success centre docs don't make it very clear and look like they relate to the older versions with the FSDB still, What i'm trying to figure out is if the NTA DB is now SQL do i need to make sure 1433 is open between the new DB server and all the other component servers or just some?

Currently everything uses the SWIS port to get to the FSDB and if it now requires 1433 i may run into issues with our security team for opening a SQL default outside of across our WAN.

meech​ If the data stays within Solarwinds then i can probably provide some feedback on the upgrade process too

dsimpkins please keep a running list of issues with success centre docs and send to me directly in email at kellie.mecham@solarwinds.com  We don't often get fabulous feedback about improvements to docs, but, we have a team of awesome people who can update mistakes--send all issues, confusions mistakes in docs to me for NTA and I can forward accordingly.  And--checking in  to get an answer to your question!

Thanks Jason,

I've seen this already but where is falls short is for example: "port 17777 is required between the main poller and the SQL database." The latest version doesn't use the fsdb application so why is this port still needed if it's just a std sql db server.

I have multiple pollers separated by a number of firewalls that are out of my control so don't want to go Swiss cheesing the firewalls with unnecessary port openings hence asking for clarification on what is required for old versus new.

Thanks Kellie,

Will drop you an email shortly.

does the upgrade intsall the NTA DB in the current SolarwindsOrion DB, or does it create a separate DB.  I am working to get my Dev environment upgraded this week to NTA 4.4 for testing.

It creates a separate DB for flow. The good news is: we can support (and recommend) placing both the Orion DB and the Flow Storage DB in the same SQL instance. If you have Log Manager, you can run that database in the same instance, as well.  We've just completed performance testing, and we're updating our documentation to reflect these recommendations now.

joer

Ok, thanks.  I asked because as I was upgrading our instance, on the DB screen, it just listed the existing DB, and wouldnt let me enter a new DB.   Said I didnt have access to add, guess I need to go back a screen and enter the SA credentials.

Somehow, The installer thinks I installed the product, even though I did not.  I can not get it to force a re-install of NTA 4.4 and the new DB.  I looked at DB manager, and only see the SWOrion DB listed.  is there a way to force the unified installer to install again?

jreves​, please update when this documentation is available.  This will be huge for several of my customers.

Hi Team,

Running an install at the moment - General experience is excellent meech

Had a bug during when connecting to the SQL database - Servername, username, password details were well known and set in stone. Connection to the DB failed however due to an authentication problem. Thought that was strange, rechecked and reentered all details a few times, no luck. Called my DBA, troubleshot, created a new account to authenticate with and again the connection failed due to authentication failure. A few attempts later I pressed back and forward on the installer and everything worked as intended.

Hi there, not my thread but seems like a good time to chime in and complete the chain:

3 actions that'd cover most of the my use case for netflow alerting:

  • Alert me when traffic between x and y is more than 40% application/port z

          (w/ standard deviation support perhaps?)

          (Alert me when traffic is mostly backups/netflix in working hours)

  • Alert me when top 5 traffic applications/conversations have changed (averaged over time)

          (This link should be used by File Transfers > HTTPS > HTTP > Email > AD sync. Anything else is weird, let me know)

and most of all clean integration with NPM alerting:

  • Alert me when interface x is >90% utilized
    • Email action: Body: <img src="//x/orion/charts.aspx?chartname=WhatsNetflowSayIsEatingAllTheBandwidth">

huh, that is a weird one.  What specifically gets us an 'excellent'  to date, adam.beedell​?  Where do we still need to work on things?

The old timey installer process was a nightmare in a multi-server environment. The "new" installer cleared that right up and I'm very grateful for it!

The pre-flight checks have been getting better and better too, and take much of the work out, again much appreciated there.

Generally speaking the UI is straightforward, clean and modern-looking, and the UX is fairly pleasurable as far as an a big 'orrible patch day does.

To improve (not sure how much is in your area):

  • Running the DB config and website config on each APE feels like it's redundant - not sure if it actually is, but it feels that way at least
  • Got thrown off in the previous patch day with the NTA DB requirements differing from the rest of the equivalent packages of the same version
  • Potentially could we run the installers on the polling engines through the primary server installer?
  • Enabling SSO during the website config has a few knock-ons and a warning would be appreciated there (as far as I understand it so far)
  • It looks like hotfix installs (and maybe later patch installs?) are mandatory, which is generally fine but is a bit of a headache when writing change control documents
  • I did get a couple more errors during the install below, I've checked the website and the maps though and they appear to be working as intended (at least at a cursory glance)
    • Failure importing sample map
    • Web request for /orion/login.aspx failed - 404

That's probably about it as far as the installer goes

As for netflow - I've had serious headaches with the UI there for some time, but i've not had a chance to play around with the new version yet, so I'll get back to you on that in a few weeks if you want?

Adding serena​ to this thread to comment on the installer feedback!

adam.beedell  wrote:

The old timey installer process was a nightmare in a multi-server environment. The "new" installer cleared that right up and I'm very grateful for it!

The pre-flight checks have been getting better and better too, and take much of the work out, again much appreciated there.

Generally speaking the UI is straightforward, clean and modern-looking, and the UX is fairly pleasurable as far as an a big 'orrible patch day does.

To improve (not sure how much is in your area):

  • Running the DB config and website config on each APE feels like it's redundant - not sure if it actually is, but it feels that way at least
  • Got thrown off in the previous patch day with the NTA DB requirements differing from the rest of the equivalent packages of the same version
  • Potentially could we run the installers on the polling engines through the primary server installer?
  • Enabling SSO during the website config has a few knock-ons and a warning would be appreciated there (as far as I understand it so far)
  • It looks like hotfix installs (and maybe later patch installs?) are mandatory, which is generally fine but is a bit of a headache when writing change control documents
  • I did get a couple more errors during the install below, I've checked the website and the maps though and they appear to be working as intended (at least at a cursory glance)
    • Failure importing sample map
    • Web request for /orion/login.aspx failed - 404

That's probably about it as far as the installer goes

As for netflow - I've had serious headaches with the UI there for some time, but i've not had a chance to play around with the new version yet, so I'll get back to you on that in a few weeks if you want?

About your comment about running the installers through the primary server installer, I have great news for you! We're working on this as a feature that we call "Centralized Upgrades" You can see this reflected on the What We're Working on for NPM (Updated June 1st, 2018)  roadmap

I'm about to do this from 4.2.3 using the Fastbit db to SQL 2016 SP1. Currently we have NTA 'installed' on the NetFlow server (meaning, the NetFlow Configurator runs on the NTA db server).

Do we do the same thing with the SQL server when we upgrade to 4.4? It seems odd to have other software running on a SQL server.


Hi Kwilson, NTA software installs on the web server/pollers, the DB server becomes just a repository

Thanks.

Has anyone tried to add a new additional polling engine with the new version of NTA installed in their environment?  I am running into a problem of not knowing the SQL credentials for the NTA database.  During the upgrade to version NTA 4.4 I was not prompted to enter creds so the install must have created these.  The Login ID that is filled in while running the config wizard on the new additional polling engine is - SolarWindsNtaDatabaseUser.  If anyone has any insight to this issue please let me know. 

On the first screen of the config wizard click the load details from Orion server button rather than clicking next next next...

This populated all the details for me when i added some new APE's last week.

Thanks for the quick reply but I do not have that option.

i can't check my system right now but i wonder if because you have tried once and created the SWNetPerfMon.db file it doesn't then give you the option again.

Might be worth finding the file and moving it out the way to see if you get it a second time round

Version history
Revision #:
1 of 1
Last update:
‎05-30-2018 11:30 AM
Updated by: