The Evolution of Transaction Management – Part 1

lanir-shachamTransaction management has evolved to a fourth generation. Based on a podcast by Doug McClure, this article covers the end-to-end transaction management evolution.

Information technology contributes to business efficiency like nothing else. Managing complex IT systems requires constant innovation; systems management has come a long way since the days of “Big Iron.” The fourth generation of systems management, widely referred to as “end-to-end transaction management,” enables IT to reach levels of maturity and stability like never before.

Mainframe Systems Management – Generation 1

The birth of application performance management tools came with the development of the mainframe Omegamon and Tmon. These tools monitor program names, Customer Information Control System (CICS) jobs and Job Control Language (JCL) tickets. As far as application performance management goes, this was really the first transaction performance management solution where everything was centered on the mainframe.

Client/Server – SQL Management – Generation 2

Next up was the client/server. Back then, the server was essentially the database. The database was the single point of failure, since it became the place where all of the data was stored and SQL queries were everything. If you had a problematic SQL query, then you had a slow transaction.

That is really when the true application performance management market emerged; with companies like Precise and Quest. The first tools for database monitoring utilized the SQL command line to extract system information. These first application performance management vendors built smart agents that collected all of the information all of the time and sometimes it actually worked in production. These application performance management tools had a nice (relative to the time) dashboard and suddenly there were viable tools on the market for optimizing the database. Database administrators finally had something to work with that gave them value. This was around 1997, when Windows 95 was just becoming stable.

J2EE Profilers and the User Experience – Generation 3

G3 is the generation that is currently coming to a close. This generation began with the emergence of Java application servers. Java’s rise within large organizations was seen in 1999-2002, and suddenly many application vendors started working with Java. As they built applications and sold them to enterprises, the number of Java application servers exploded. This phase introduced the need for a Java profiler tool that could manage performance. Java profilers were brought to the market by Wily, BMC and others.

At the same time, with the rise in online computing, it became important to track user experience; your Website was suddenly your business. With the Web so critical to doing business, it was important to make sure it was performing as expected. At the time, running synthetic transactions with Gomez, Mercury Topaz, and others in order to ensure availability was the way to do it.
Alongside the JVM profilers and user experience tools, there were always network probes to ensure the network was working. As TCP/IP became the standard, solutions parsed and sniffed these protocols, parsing network traffic in order to monitor performance.

What Marks the End of 3G Systems Management?

In the last few years, the last two major silos that we have not yet listed, middleware and storage, have had tools developed for them. For example, companies like Onaro (acquired by NetApp) developed tools that extended application performance management into the storage tier and MQSoftware (acquired by BMC) developed tools for Websphere MQ. Today, every silo has its own performance management tool, all with the objective of achieving greater application performance.

All of this activity happened over the past 6-7 years. Every silo was developed on its own and was embraced by the industry. In the past two years, the market become stable; however, we do see new concepts for application servers, and other languages—not just Java-based–and we see a lot of integration. Web services is the new buzz word, which is really just an easy way to do integration (enterprises were doing HTTP-based XML 10 years ago). All of these changes notwithstanding, there are no new silos. There are more of them, but not many different types of silos.

Stability and Maturity = 4G End-to-End Transaction Management

The market has matured to a level where you can deploy end-to-end solutions. If you would have tried to do this a few years ago, you would have run into complications. Currently there is enough stability in new technology as far as enterprise applications are concerned in order to enable the proliferation of end-to-end management. Of course, there is always constant progress in storage and network throughput. These technologies will continue to move forward, but the technology for tracking a transaction is here to stay. TCP/IP will be with us for the next 10-20 years, and HTTP protocols aren’t going to change in the next 5-10 years. While Java will not go away as an application server, there will be new methods for implementing an application server. It will not necessarily be JVM, it could be a new process; however, eventually it’s going to be some sort of process, multi-threaded or not, which will handle the transaction. The overall architecture of the application is going to stay more or less the same: a rich, mixed, and complex topology that is difficult enough to manage as it is. This maturity is exactly what is enabling transaction management to become the fourth generation of IT systems management.

Bringing IT Together

For the moment, forget the buzz words like Business Service Management, Business Transaction Management, Service Level Management and others. What we are talking about is the evolution of IT systems management, which is defined as everything that is needed in order to make the IT data center work. Transaction management will compose 70% of the systems management space. It will be critical (and already is) in ensuring sure that your data center actually works.

A big thanks to Doug McClure of for creating an informative podcast with Correlsense CTO Lanir Shacham from which the ideas for this article were taken. Take time to listen to Doug’s podcast, which aims to shed light on the next generation of IT systems management.

SharePath by Correlsense is a pioneer in 4G systems management, and can help ensure optimal performance of your complex, distributed, applications.