66 Replies Latest reply: May 20, 2013 7:57 AM by ctopaloglu RSS

    What we are working on for NPM

    mavturner

      Now than NPM 10.2 is out the door, development has been busy working on the next release. Below are some of the features we are working:

       

      1. Background research and prototyping for new charts (there will be a blog post describing this in more detail soon).
      2. SNMPv3 support for AES-256.
      3. Improved handling of devices with multiple IPs
      4. Better support for international customers.
      5. Enhanced alerting around UnDP table pollers.
      6. Native support for Alaxala and Apresia (CPU, Memory, and Interface utilization).
      7. Support for changing polling method (for example: from WMI to SNMP).
      8. Continued improvements to ConnectNow (leverage CDP and LLDP information).
      9. Incorporating this How long each interface or node was down. by Gob which shows how long a node or interface was down.
      10. Support for a new SWIS Clear Event verb (helpful for integration with 3rd party ticketing systems).
      PLEASE NOTE:  We are working on these items based on this priority order, but this is NOT a commitment that all of these enhancements will make the next release.  We are working on a number of other smaller features in parallel.   If you have comments or questions on any of these items (e.g. how would it work?) or would like to be included in a preview demo, please let us know!
        • Re: What we are working on for NPM
          smargh

          5) Oooooooooooooooooooooooooh yes yes yes. Table-based alerts do need improvement - it's very awkward at the moment. Alerting based on other values for that row in a different column would be awesome, but we would also need to be able to use other row values in the alert emails/actions.

          I'm not particularly bothered about the others, except for WMI<->SNMP switching which would be useful. There are a few things, however, that would significantly improve day-to-day duties with NPM:

          11) A more user-friendly method of bulk "schedule unmanage" - the current utility is very much a bolt-on & isn't integrated into the product.

          12) A method of auto-selecting a working mail server to send alerts through, rather than having a choice of only one. This is a very complex suggestion though.

          I use the Windows built-in evntwin.exe / evntcmd.exe for sending events to Orion as traps, with a small Powershell scripts for bulk SNMP config updates to servers:

          13) A more user-friendly conditions screen. It's very clunky right now, i.e. no "copy", import/export, the screen refreshes when adding a row, etc.

          14) The varbind numbers changed (minus 2) with a recent update - i.e. vbData6 became vbData4, vbData9 became vbData7. Needs consistency among versions.

          15) For trap alert emails, I should be able to use the varbind label, i.e. eventText or eventVar1 - these hopefully should never change and would significantly aid readbility of alerts.

          16) Built-in parsing of evntwin.exe-based SNMP traps would be truly awesome. Preparing these alerts & selecting criteria is massively time-consuming and quite prone to mistakes.

          17) Trap alert throttling in the form of only executing one action per host every X minutes rather (or in addition to) than the current options. Perhaps have an option to discard traps being sent to Orion at a high rate - for example, some Exchange events can number in the thousands, and the built-in Windows SNMP trap throttling system is awkward.

          18) A method of testing ("test firing") a rule to make sure it will work.

          • Re: What we are working on for NPM
            byrona

            I am very excited about #1, I have made several suggestions and feature requests on that one; can't wait to see the blog post!

            I am also very excited about #3 as many of us have been asking for that for years.

            Overall I think this is a great list of things to look forward to, thanks for the update!

            • Re: What we are working on for NPM
              jawells

              Hi

              Please add better discovery filters to the next release as this has been on the cards for years.

              1: Ability to filter by hostname or interface ifAlias

              2: Ability to filter by interface type or vendor

              3: Ability to add to a group based on a filter

              4: Ability to remove hostname suffix

              5: Ability to add custom properties based on filtering, meaning if your filter matches then add these custom properties to the node or interface

              Keep up the great work

              Kind Regards

              James

                • Re: What we are working on for NPM
                  jonchill

                  How about improvements around report writer and being able to graph reports and include a lot more flexibility.

                    • Re: What we are working on for NPM
                      mavturner

                      Thanks for all of the great feedback everyone.

                      jonchill, although we aren't going to work on Report Writer and graphs in reports, it's something currently under investigation to work on soon. 

                      jawells, yes we definitely need to improve the discovery filter options. We took some of this feedback into account when we built UDT, but we aren't currently focusing on this problem today.

                      Keep the feedback coming!

                      Mav

                        • Re: What we are working on for NPM
                          rgward

                          yes we definitely need to improve the discovery filter options. We took some of this feedback into account when we built UDT, but we aren't currently focusing on this problem today.

                          :|  I was hoping there would be at least some improvements in discovery filtering in the works for the next release with all the complaints and feature requests from the user community.  Due to lack of discovery filtering we've never used this feature of NPM.  The lack if this functionality is preventing us from abandoning our HP OpenView product. Our NOC has been screaming for this improvement in NPM since this feature was released in 10.0 so they don't have to continue to maintain OpenView just for it's discovery capabilities.  PLEASE give resources to improving this feature for the next release!

                          • Re: What we are working on for NPM
                            jawells

                            Hi Mav

                            I dont like banging on about the discovery filter as it is something that has been requested so many times, but if SolarWinds really want to progress into the automation aspect of the product, this is the primary key if I can put it that way. As some customers and I am talking about large customers, already have a CMDB external to NPM and automation is at the forefront of their process. having the discovery filters is key to loading what should and should not be in NPM/NCM/UDT as the manual effort is painful for businesses that have large environments.

                            One thing I have noticed with the developement of the product/products is you are keen on new features but not keen on improving existing features as some of the requests like this one are 3 to 4 years old. If I might be bold to say practice what you preach, you claim to listen to users, then this is someting we have all being crying for a long time.

                            Sorry for the rant ,I really like the product/products as I have come from the big enterprise toolsets , like CA eHealth, Openview, Ciscoworks, VitalNet, Omnibus, SMARTS and so on and there are some basics like discovery filtering that should be in your product from day one to be honest.

                            Kind Regards

                            James

                              • Re: What we are working on for NPM
                                sirpaw

                                Hi Mav,

                                #3 is going to solve what has been a pain in the **** for years, hope all of these comes into fruition! also feeling good about #1 and #9, keeping track of the downtime age will be quite helpful..and being outside the US, i hope something great is in store for us with #4.

                                I have to agree with jawells as well, there's room for improvement in discovery filtering. But all in all, really looking forward to all of these. Thanks!

                                • Re: What we are working on for NPM
                                  michal.hrncirik

                                  Hi James NPM 10.5 finally has it

                                    • Re: What we are working on for NPM
                                      ctopaloglu

                                      Hi Michal,

                                       

                                      What about Multiple IP support polling for single node. We have been waiting for many many years for this feature. Even basic free NMS solutions have it.

                                        • Re: What we are working on for NPM
                                          RichardLetts

                                          What do you mean by "Multiple IP support polling for single node" -- if you mean able to poll the interface IP addresses then why is this a good idea? In most networks on routers one normally assigns an ip address to the loopback interface, and then use routing protocols to distribute that out: You don't need to be able to poll SNMP interface using the interface IPs. If you can't poll the loopback address then there is a routing issue, and that needs fixing.

                                            • Re: What we are working on for NPM
                                              ctopaloglu

                                              Hi Richards,

                                               

                                              Loopback is not a solution for us. Let me give you an example.

                                               

                                              Let's say we are using a VPN device on our edge office. This edge VPN has two connections. Primary one is ADSL and the other one is 3G for backup. Both have dynamic IP addresses (unfortunately).


                                              • We can't monitor it with loopback IP because the tunnel may go down but the device can be up and running. And we have to know the root cause.
                                              • We can't monitor it with static IP because the IP can be changed (Dynamic). Of course we prefer static IP but not every static IP is cheap on some countries. We are working on a dynamic IP address solution on our side. But that's another story.
                                              • Also we can't monitor the static IP because it can be up but the loopback (VPN tunnel IP) is down. We need to monitor tunnel as well.
                                              • On some situations the static IP address is up and the interface (VPN tunnel IP) seems up but it is actually down. You can't ping the VPN tunnel interface (Juniper SRX Series have that bug). And opening a CASE is not always a solution .
                                              • In addition to these we need the availability of each connection (both primary and secondary). That's important for us to decide which connection is better.
                                              • Lastly, routing issues not always on our side. We have 400+ offices all over the world. And most of the time the ISPs are to blame

                                               

                                              NPM v10.4 comes with routing support. Maybe we can use this to track the availability of primary and secondary connections. But currently is it not possible on the NPM or takes too much of our time. So do you have any other solution beside "Multiple IP polling support for single node" for us? Here is idea for this. Please vote if you think this is important.

                                               

                                              Multiple IP Polling Support for Sing Node

                                               

                                              Thanks.

                                  • Re: What we are working on for NPM
                                    jasadelll

                                    +1 on discovery filters.

                                    • Re: What we are working on for NPM
                                      herwig

                                      I also vote with urge for useful discovery filters,
                                      at the moment discovery is producing more work than benefit!

                                      when I add a node manually, I select my 2-3 interfaces and fine --> 1 minute
                                      when I use discovery e.g. for our routers with voice modules (PVDMs) I have >300 interfaces to ignore for every node --> click to death
                                      when I use discovery for stacked switches, I have >300 interfaces to ignore for every stack --> click to death

                                      thanks for hearing us
                                      Herwig 

                                        • Re: What we are working on for NPM
                                          shawnaz

                                          Discovery is single handedly the biggest feature holding us back from really getting use out of NPM.  Currently everyone has this perception that we can't trust that Orion says everything is good because we've been burned too many times by not having something in Orion (due to engineer's fault for not putting it in there, obviously, but making discovery useful would provide peace of mind that we lack)

                                           

                                          Auditing who adds and especially deletes nodes would be a superb start for audit.

                                        • Re: What we are working on for NPM
                                          ctopaloglu

                                          I love it James. These enhancements are great! Please add my vote to all them.

                                        • Re: What we are working on for NPM
                                          jspanitz

                                          YES!  #1, #3 and #5 are very much appreciated.  Have been looking forward to #1 for quite some time.  Please go big on this one!

                                          Would like to see more JUNIPER integration.  As in Juniper SA / MAGs don't poll CPU / Memory OoB.  Would also like to see a better UnDP interface. - ie, when trying to build and test a UnDP.

                                          • Re: What we are working on for NPM
                                            mdriskell

                                            #3 is #1 in my book and has been since I first installed Orion 6 years ago.  The forums are filled with posts from myself and others of the complicated workarounds we implement to handle this problem on our own.

                                            The other one that I would hope could be done but is not on this list would be the ability to add some copy or import features to the traps and syslog modules.  I can copy an alert and alter for a new one but every one of my trap rules has to be written from scratch even though I may be only altering one simple aspect of it.  

                                            It seems a little odd that even though the interfaces are very similar between the alerting, SNMP, and syslog filters the features are not constant in the least.

                                            • Re: What we are working on for NPM
                                              ecornwell

                                              Nice list.

                                              I would still like to see some work be done on the custom properties page.

                                              IE:
                                              Be able to set a list of drop down boxes for pre-entered values to prevent someone from using the wrong values.
                                              True/False being a checkbox
                                              Ability to define a "label" for the custom property that can be used to explain what its for.

                                              • Re: What we are working on for NPM
                                                smartd
                                                Very Excited about #8 and #9.  We have a daily network meeting, and the resource from Gob is what is used for discussions.  We would like to be able to sort the summary list by the outage count or outage time.  Also excited about LLDP leverage since all our HPs and Juniper support it.
                                                • Re: What we are working on for NPM
                                                  raj512

                                                  I'd like to submit a couple additional feature requests:

                                                  The abiliity to "search" within groups and dependencies

                                                   

                                                  And most importantly - AN AUDIT TRAIL IN NPM so you can see who is adding/removing/changing nodes

                                                    • Re: What we are working on for NPM
                                                      dquerry

                                                      I agree with raj512. We need an audit that shows who is adding/deleting/ interfaces and nodes!

                                                      • Re: What we are working on for NPM
                                                        mavturner

                                                        Thanks Rachel, the audit trail is definitely something we get a lot of requests for. Can you tell me which specific events are the most important for you to audit? For example: add node, delete node, un-manage node, add/delete interfaces, add/delete accounts, etc. What are the most important events for you to audit?

                                                        Mav

                                                          • Re: What we are working on for NPM
                                                            byrona


                                                            Thanks Rachel, the audit trail is definitely something we get a lot of requests for. Can you tell me which specific events are the most important for you to audit? For example: add node, delete node, un-manage node, add/delete interfaces, add/delete accounts, etc. What are the most important events for you to audit?

                                                            Mav

                                                             



                                                            Hey Mav

                                                            Having this audit trail is really important to me as well so I hope you don't mind me responding to this question as well...

                                                            I think for any audit trail to be successful it needs to track any and all changes made; what was changed, who made the change and when it was made.

                                                            If you are looking for a place to start then I would go with any changes made to nodes in Orion.

                                                            I am very interested to see what others think about this!

                                                            Hope this helps!

                                                            • Re: What we are working on for NPM
                                                              ctopaloglu

                                                              For example; If a node gets deleted by mistake there is no way to add it again unless you remember it's IP address. This is not acceptable for us. Audit at least shows the IP address of the deleted node and also who, when, and WHY.

                                                               

                                                              Yes the WHY feature is also important for us. If it can show who and when only that's not good for us. Because you need to ask him/her why he/she did it. Remember if you try to shutdown/reboot a windows it asks for a reason (shutdown tracker feature). Last but not least the unmanage functionality is also need this WHY field. I have created a custom property (Unmanage_Reason) just for this very specific reason. They unmanage every down node for no reason!

                                                            • Re: What we are working on for NPM
                                                              taylorfc

                                                              Ditto on the audit trail!

                                                            • Re: What we are working on for NPM
                                                              sja

                                                              WAOO

                                                              SW start deliver :-)

                                                              • Re: What we are working on for NPM
                                                                mvjames

                                                                how about fixing the "groups" and "dynamic groups" funtionality within the account limitation, described in the following post:

                                                                Node groups - account limitation issue

                                                                 

                                                                Not sure how useful filtering out all other groups (not the members of those groups) vs the expected functionality of only showing what is within the selected group....

                                                                • Re: What we are working on for NPM
                                                                  henriknoerr

                                                                  Hi,

                                                                  Nice with an update! - I really like the work being done on UnDP table poller alerts and better graphs!

                                                                  As already mentioned I would also like an audit trail, furthermore the ability to have multiple nodes with a duplicate IP would be a dream come true.

                                                                  Duplicate IPs is necessary for polling the same node with different snmpv3 context values.

                                                                   

                                                                  Cheers,

                                                                  Henrik Noerr

                                                                  • Re: What we are working on for NPM
                                                                    byrona

                                                                    As more and more stuff is added to Orion (via NPM, APM, etc) I find more and more great resources getting added to my node view.  While I love having all of this extra information I am also finding that my node views are becoming a bit cluttered feeling.

                                                                    I would like to see SolarWinds (and their community) put some thought into added functionality that could help clean up these views.

                                                                    I had posted a feature request a while back (that I can't seem to find now) on having the ability to expand and collapse resources within a view and having the ability to specify on a per resource basis if they defaulted in an expanded or collapsed state.

                                                                    I would love to hear other ideas on functionality to accomplish cleaning up these views without removing useful resources!

                                                                      • Re: What we are working on for NPM
                                                                        beijota2

                                                                        I want to add a +1 to Byron's request and point out that it would be very helpful to make the collapsed resources not load until they are expanded so the node details page loads.  This would be extremely helpful with Netflow charts.

                                                                         

                                                                        I love the new polling engines/details page, especially the change to the polling rate/server capacity spec.  You wouldn't believe the peace of mind this has given my boss knowing exactly where we stand as far as monitoring capacity.  It's also great to see almost instantaneously how polling interval changes affect the capacity of the server.

                                                                         

                                                                        I would like to request that on the polling details/engines page that the last database sync result be changed back to a date/time format rather than xx seconds ago.  I used this number in the past to make sure that I was looking at fresh data and to make sure that the page was actually refreshing.

                                                                         

                                                                        I love this new release and i'm looking forward to the next!

                                                                      • Re: What we are working on for NPM
                                                                        nsoc

                                                                        Does #7 mean we can have a nodes status by SNMP instead of ICMP?

                                                                        • Re: What we are working on for NPM
                                                                          tmeyers

                                                                          Hi.... any chance that the ability to monitor virtual CPU allocations on IBM P-series (P595, P780) AIX servers is in the cards soon?

                                                                          Thanks!

                                                                          NPM 10.2.2

                                                                          NTA 3.8.0

                                                                          SEUM 1.5.0

                                                                          IVIM 1.2.0

                                                                          SAM 5.0.0

                                                                          IPSLAMGR 3.5.1

                                                                          Storage Manager 5.1.2

                                                                          • Re: What we are working on for NPM
                                                                            smartd

                                                                            Feature: Discovery add interfaces with multiple MAC addresses.

                                                                            There's a perl script that does this on Content Exchange.  Perfect.  I want to automatically add any interface that has three or more MAC addresses associated with it. (Three because we use voip phones with a one port switch presenting two MACs).

                                                                            That would catch all uplinks.  Exceptions from LLDP could then be added.

                                                                            • Re: What we are working on for NPM
                                                                              Bain_606

                                                                              Please, just please bring back #7 - I am drowning in duplicate nodes with alternate IP addresses that are ALL POLLING SNMP.  

                                                                               

                                                                              • Re: What we are working on for NPM
                                                                                Radioman

                                                                                Maybe I've missed it somewhere, but all the refs to IPv6 on search are from 2011 or older. Is the newest version of NPM 10.2.1 IPv6 ready? are all SW products ready for it?

                                                                                Network to FVO IPv6 3Q12.

                                                                                Thanks

                                                                                Mike

                                                                                Verizon

                                                                                • Re: What we are working on for NPM
                                                                                  JaroslawLadyga

                                                                                  Hi Mav,

                                                                                  the list has great improvements but there is still lack of few features I'm waiting for. I must agree with jawells saying "you are keen on new features but not keen on improving existing features".

                                                                                  Here are few examples:
                                                                                  1. Report Writer - I see that it has not been improved for years and it is practically the same application now as it was in 2006 when I started with Orion. Few new report templates for APM and NTA are the only changes done .
                                                                                  There is really huge amount of data in Orion database but it is very hard to analyze historical data without integrated tools for that. Report Writer has too poor functionality for that. I see very often that I can not get what I want from Report Writer and I must export data to Excel format file - transfer it from Orion server to my PC and work on the spreadsheeet manually. Connecting Excel with ODBC to database of around 10 GB from remote location ... forget it - would spend hours for transmission. And finally I'm not SQL specialist and have no budget for SQL consultant to make me SQL querry based report.
                                                                                  Report formatting function is very poor as well. There is practically one layout. No option for include graphics and diagrams.
                                                                                  2. Graphics and diagrams - I would say that is even step back when System Manager disappeared forever. I was able to adjust diagrams with subject, time frame, scale, legend and size in seconds or few minutes by System Manager. I must spend tens on minutes now to get the same set of diagrams.
                                                                                  3. Network Atlas defaults - that is something I asked over year ago. Is it really so difficult to enable setup of default line thickness, color or font type and size?
                                                                                  4. Advanced Alerts - still lack of dependancy on another alert.
                                                                                  5. Orion website users log - promised over year ago - called an audit trial !
                                                                                  6. Orion website - node management - resources - when virtual server got reconfigured old volumes have the same color as the new ones and it is necessary to compare IDs to recognize which disappeared and which are active. I asked for that already year ago.

                                                                                  Orion Core 2011.2.2, APM 4.2.0 SP1, NPM 10.2.2, NTA 3.8.0, IVIM 1.2.

                                                                                    • Re: What we are working on for NPM
                                                                                      lord_haw

                                                                                      One thing I'd like to see would be the ability to both turn off device rediscovery and adjust the rediscovery interval for an individual node.  The reason for this is because we've run into an SNMP based issue on a couple of devices that only causes issues when a rediscovery takes place.  The regular poll doesn't affect it.  So if we could turn off rediscovery for those two individual devices or at least adjust the interval to be very high for them without changing it globally it would be nice.

                                                                                    • Re: What we are working on for NPM
                                                                                      rickg

                                                                                      I asked for this previously....

                                                                                      On error reporting, giving me an number of errors in a certain time frame is OK, but not especially helpful.  I need an option to convert the number of errors into a percentage of errors based on the number of packets processed.  Seing 1,000 errors is nice, but telling me that 15% of my packets contain errors is useful.  It's data I can take to the carrier and get them moving towards a solution.  NPM already collects the data, so how about a new and useful way to display it?

                                                                                      Rick

                                                                                      • Re: What we are working on for NPM
                                                                                        Dal

                                                                                        What about some proper map handling finally? It has been asked about countless times.

                                                                                        Scalable maps, scrollbars in map window, etc.

                                                                                        Please please implement native support for mikrotik / witelcom radios!!

                                                                                        • Re: What we are working on for NPM
                                                                                          Spiky

                                                                                          how about (since 10.2)  stopping the database maintenance each night,  deleting from the logs for any node that has been removed

                                                                                           

                                                                                          as  now, we can no longer run reports on nodes we have removed, for us it's a financial issue

                                                                                          • Re: What we are working on for NPM
                                                                                            ctopaloglu

                                                                                            I am using the latest beta (3) and I did not find anything about the multiple IP address feature anywhere.

                                                                                            • Re: What we are working on for NPM
                                                                                              ctopaloglu

                                                                                              Some of our nodes (VPN Edges) lost their ping for sometime. So If a node respond to SNMP but not ping it is proves that node is not down.

                                                                                               

                                                                                              Also if a node is unplugged and other device with same IP address (but different sysoid or no snmp at all) came in. It still shows the device is up! That's not a true. We didn'tt have a solution  for this.

                                                                                              • Re: What we are working on for NPM
                                                                                                buzzwork

                                                                                                Are there any news on Trapeze thin wireless AP/controller monitor ? like your Cisco wifi today

                                                                                                • Re: What we are working on for NPM
                                                                                                  ctopaloglu

                                                                                                  Linux (net-snmp) OS breakdown support! I don't want to see just net-snmp anymore. Also I would like to see which linux and which version as well.

                                                                                                   

                                                                                                  Ex; Red Hat Enterprise 6 (32 Bit)

                                                                                                  • Re: What we are working on for NPM
                                                                                                    andrewbrill

                                                                                                    I would think the number one priority in all of the goals and objectives for Solarwinds would be to work on performance and scalability and focus on data architecture across the product lines.  I believe there are multiple areas where if the data architecture were the primary focus, many other things could be possible.  Several of the stored procedures for alerts/alarms, ability to retain data for longer periods in the original retention polling periods, etc.  Meanwhile, the concept of insuring one can "Retire" and interface and so continue to leverage the data.  This is true of so many of the modules but Orion NPM is at the core of it.  Meabwhile ensuring that Web front end is compatible with the modules is key to performance and scalability.

                                                                                                    • Re: What we are working on for NPM
                                                                                                      rmoney

                                                                                                      I second the comment by ecornwell. We absolutely need to be able to specify custom properties as dropboxes, so an extra space doesn't create a whole new property.

                                                                                                      • Re: What we are working on for NPM
                                                                                                        ray.rechtin

                                                                                                        I would like to add another voice to the request for an audit trail.

                                                                                                         

                                                                                                        One thing I would love to see is more granular and well-built permissions.  If I give someone access to manage accounts, they should be able to do all tasks involved therin - including managing the limitations on the account - instead of also having to give them the 'customize views' permission and risk them altering the default range of graphs that all our company sees.  I would love to be able to give people access to add new devices without also allowing them to delete devices.  I would love to give people access to edit existing devices without being able to add or remove.

                                                                                                         

                                                                                                        I would also love to be able to define regex/patterns for custom properties.  we have device naming standards for sanity, but there is no way for us to enforce them within the application.  We can run scripting against the database to find the ones that don't work, but then to get to root cause, we have no audit trail to tell us who it was that didn't follow standards.

                                                                                                        • Re: What we are working on for NPM
                                                                                                          ctopaloglu

                                                                                                          Hello to all,

                                                                                                           

                                                                                                          Some of the important Roadmap items are not met unfortunately.  Some of them are...

                                                                                                           

                                                                                                          • Multiple IP Address for single node.
                                                                                                          • Auditing
                                                                                                          • Re: What we are working on for NPM
                                                                                                            AMSW

                                                                                                            additional authentication methods (LDAP, LDAPs, TACACS, etc) for user accounts on the web console???  this is years overdue and we've been a customer since the pre-7.x releases and will probably end up dropping as a customer because of this missing functionality...

                                                                                                            • Re: What we are working on for NPM
                                                                                                              jerquiaga

                                                                                                              Overhaul the web interface! With lots of monitors being widescreen and super high resolution now, it would be great to be able to show more than three columns on any given display page.

                                                                                                              • Re: What we are working on for NPM
                                                                                                                macgyver

                                                                                                                Please give us better Email alerts, with easy ways to handle acknowledgements, and a way to interface with third party ticketing/call systems.