There is a reason why resolving application performance issues is so hard. Actually, I think there are a lot of reasons, but I’m going to spend a few posts looking at one set of reasons.
A lot of human endeavors are linear: move in one direction until you reach your goal. Troubleshooting application performance is non-linear. It has a lot of different, yet interrelated dimensions. The only way to resolve application performance issues is to address the dimension that is limiting performance. But with the interrelationships between the dimensions, the symptom may be far removed from the constraining dimension. The cause of a performance issue can be a long way from its visible effects.
One dimension is the performance of the user interface. Whether it is a native application or a web browser, every application has a User Interface (UI). Since the UI is close to the user, it is what they perceive as the application. Something as simple as a bug in Adobe Flash or a laptop that hasn’t been rebooted in months can lead to a poor user experience. Of course, the application that is slow for the user may not be the cause. Another application taking up 99% of their laptop’s CPU can be the bully, the application that they are trying to use may be a victim. Just inside the user’s computer, there are multiple performance dimensions.
The next dimension is the network between the user’s device and the application servers. From our desks outside the data center, the network can be very fast. But if your users are on the end of a busy WAN circuit, or over a VPN, or half way around the world,j then they will have a very different experience. What about load balancers, firewalls, and even simple things like name resolution? Having an out of date primary DNS server can add a minute to the application’s initial load time. The network adds still more dimensions to the troubleshooting problem.
Then there are dimensions on the application server. Does it have enough RAM and CPU? Are its disks up to the load of the application? What about the optimization of the operating system? Have the drivers for the storage and network adapters been updated to the latest? Or have they been updated to a faulty release? Is the cause of the performance problem a periodic scan by the anti-virus software? The application server adds more dimensions to any performance problems.
Maybe there is a time dimension? End-users in another country may access the application server at an unusual time. Are there maintenance tasks happening that might make the application slow at exactly the time that your European users need to get their end of day processing done? With all these possible dimensions to be investigated, it is no surprise that performance troubleshooting can take some time.
Performance troubleshooting has many dimensions. Until you identify the limiting dimension you will never fully resolve the performance problem. The number of dimensions increases substantially when you add virtualization to the mix. Next week I will look at some additional dimensions that you need to consider when virtualization comes into play.