I recently listened to a webinar by TRAC research (Bojan Simic, David Boynton) regarding “Transaction Centric approach to APM”. In my opinion, this is the most comprehensive and relevant research on the topic I’ve read in a long time. The webinar focuses on real customer challenges and does a superb job analyzing the underlying problems behind those challenges. According to the research (which included roughly 200 valid respondents from small, medium and large organizations), there are 6 main challenges which IT organizations are facing regarding performance management:
In this series of posts, I will explain how these challenges are related to each other, and could be addressedby defining the right roles in the organization and the tools to support them.
Regarding usability of performance data, respondents have reported that they have challenges with:
First, I believe there is a strong correlation between “Time spent correlating performance data (63%)”, “amount of performance data that is not relevant (61%)” and the major challenge of “time spent on troubleshooting performance issues” (pointed out by 63% of the responders). The time customers spend on troubleshooting performance issues is likely to be highly correlated with the time they spend correlating performance data, and time spent finding relevant data. This means if you solve these two problems (“find the right data” and “correlate the data”) you would likely solve the number 1 challenge in performance management – time spent troubleshooting performance issues.
Next, the research states that very few organizations can identify who is in charge of application performance.
For the majority of organizations there is no single owner. There are multiple roles scattered across multiple domains, in charge of different elements, each with his or her own tools and methods. The lack of an “owner” with a dedicated “owner tool” is why these organizations find it hard to deal with performance issues.
Now, had there been an “owner” for application performance management – what would this owner’s tool be able to do? The ultimate goal is to find which area negatively impacts end user experience. You could say that “end-to-end” monitoring tools are supposed to do that, but this is not accurate.
Having an “end-to-end” view is not enough because it does not provide the context of the end-user request within each element of their experience. Customers are often looking for “end-to-end” tools without understanding that the context is what matters. In order to “own” application performance you need to clearly see how the “end user experience pie” breaks down to each area: Web servers, application servers, ESBs, web services, databases, mainframe, and third party services.
There needs to be an “owner” in application performance. This responsibility should not be scattered. Whether this should be a performance engineering group or someone else – is a good debate, but there needs to be an owner. The second factor is that this owner needs a dedicated tool. A tool that would act as the “spinal cord” of performance management, overlaying all other existing tools. BTM, APM, end-to-end… call it whatever you want. It’s not the name that counts, but the capabilities.
When choosing your “application performance owner” tool, keep in mind that regardless the marketing terms vendors use – without user transaction contextual end-to-end view, your performance data will remain useless as it is today, and you would lose precious time, resources and money to browse through piles of useless data (61%) and try to do the impossible mission of correlating data from various sources (63%). An appropriate “performance owner” tool needs to track an individual user transaction and meter its execution time across each hop. This would address both challenges of browsing through piles of irrelevant data and correlating it across multiple tools with different terminology, metric and metadata.