When you go into a War Room situation, you have to get your priorities straight right away. Most importantly, you’re not trying to assign blame. While the problem persists, blame is pointless and a waste of time. You need to find the source of the latency. Was something changed? Has some iteration gone foul, one of thousands along the chain from the user’s click to the end transaction? Is some platform no longer valid? Has one of your vendors simply stopped supporting an old version of their software? What is going on?
Business transaction management means end-to-end monitoring. You need to be able to see into every tier of every transaction. Every click needs to be visible, so that you can become both a detective and a judge, and root out the prime suspect. Once you’ve achieved a level of monitoring where you can see into each individual link in the chain, these Friday afternoon War Rooms become less of a debacle, and more of a help. You start to find latencies before they crop up, and suddenly you can count on end user experience being smooth and painless, the way it should be.
In my last company we used to tell jokes about the network guys. They are the first to blame (it’s easy). But eventually, once all the other possibilities have been removed, it is the Network to blame – the network between teams and the people on those teams; it’s the same thing every time.
In 16 years managing IT in War Room situations, I’ve found that end to end monitoring is the only way to effectively manage a network of any kind. When you can see everything, you don’t have to worry. You don’t have to wonder if something is going to go wrong and slow you down. And then your Friday afternoons can be used the way they should be: planning your weekend!