G4 IT Systems Management is Here

Information Technology contributes to the efficiency of the human race like nothing else. Keeping up with the management of complex IT systems requires constant innovation; systems management has come a long way since the days of "big iron". The fourth generation of systems management, widely referred to as "end to end transaction management", enables IT to reach levels of maturity and stability like never before.

Mainframe Systems Management – G1

The Birth of application performance management tools came with the development of the Mainframe Omegamon and Tmon; these tools monitor program names, Customer Information Control System (CICS) jobs and Job Control Language (JCL) tickets. As far as application performance management goes, this was really the first transaction performance management solution; where everything was centered on the mainframe.

Client/Server – SQL Management – G2

Next up was the client/server; back then, your server was essentially the database, the database was the single point of failure, it became the place where all of the data was stored and SQL queries were everything. If you had a problematic SQL query then you had a slow transaction.

That is really when the true application performance management market emerged; with companies like Precise and Quest. The first tools for Data Base monitoring utilized the SQL command line to extract system information. These first application performance management vendors built smart agents that would collect all of the information all of the time and sometimes it actually worked in production. These application performance management tools had a nice (relative to the time) dashboard and suddenly you had this sexy, easy to work with tool for optimizing your database, DBAs finally had something to work with that gave them value. This was all around 1997, when Windows 95 was just becoming stable.

J2EE Profilers and the User Experience – G3

G3 is the generation that is currently coming to a close, as will be explained. This generation began with the emergence of Java Application Servers, Java's rise within large organizations was seen in 1999-2002, suddenly a lot of application vendors started working with Java, they built these applications and started selling them to enterprises and suddenly you had Java Application Servers everywhere. This brought on a whole new black box that needed a tool that could manage its performance – that tool, Java profilers, were brought on by Wily, BMC and others.

At the same time, with the dot com bubble, it became important to track user experience; your website was suddenly your business. With the website so critical to doing business, it was important to make sure that things were performing as expected; running synthetic transactions with Gomez, Mercury Topaz, and others in order to ensure availability was the way to do it.

Alongside the JVM profilers and user experience tools, there were always network probes that made sure that the network was working. As TCP/IP became the standard, you had solutions that were parsing and sniffing these protocols, parsing network traffic in order to monitor network performance.

What Marks the End of 3G Systems Management?

In the last couple of years the last two major silos that we have not yet listed; Middleware and Storage, have had tools developed for them. For example; companies like Onaro (acquired by NetApp) developed tools that extended application performance management into the storage tier (Application Insight) and MQSoftware. Today, every silo has its performance management tool, all with the objective of achieving greater application performance.

All of this happened over the past 5-6 years; every single silo was developed on its own and was embraced by the industry. What we see happening in the past two years is some kind of stability, we do see new concepts for application servers, other languages - not just java based, and we see a lot of integration - web services is the new buzz word - really it's just an easy way to do integration (enterprises where doing http based xml ten years ago). All of these changes not withstanding; you do not see new silos. You do see more and more silos, but not different types of silos.

Stability & Maturity => 4G (End to End)

The market has matured to a level where you can deploy end to end solutions, if you would have tried a few years ago you would have run into complications, right now there is enough stability in new technology as far as enterprise applications are concerned in order to enable the proliferation of end to end management. Of course there is always constant change and progress in things like storage and network throughput, these technologies will continue to move forward but the technology for tracking a transaction is here to stay for the foreseeable future. TCP/IP is going to stay with us for the next 10-20 years, HTTP protocols aren't going to change in the next 5-10 years, Java will not go away as an application server, there will be new methods for implementing an application server - it will not necessarily be JVM it could be a new process – but eventually it's going to be some sort of process, multi threaded or not, which will handle the transaction. The overall architecture of the application is going to stay more or less the same; a rich, mixed and complex topology that is hard enough to manage as it is. This maturity is exactly what is enabling transaction management to become the fourth generation of IT systems management.

4G Systems Management – Bringing IT all together

Forget the buzz words for a moment - Business Service Management, Business Transaction Management, Service Level Management and others - what we are talking about here is the evolution of IT systems management itself - which is defined as everything that is needed in order to make the IT data center work. Transaction management is going to compose 70% of the systems management space – it will be critical (and already is) in making sure that your datacenter actually works.

A big thanks to Doug McClure of for creating an informative podcast with Lanir Shacham from which the ideas for this article were taken. Take time to listen to Doug's Podcast which aims to shed light on the next generation of IT systems management.