DevOps and Metrics

DevOps is no Panacea
As you probably know, “DevOps” is a methodology that attempts to narrow the divide between developers and operations. By definition, it does no good to speed up code delivery if operations is not equipped or eager to roll out changes at an increased velocity. DevOps proposes a release manager role and increased communications to bring the parties together.

Developers and operations often have conflicting goals or at least goals that do not mesh well: developers and users want new features added to the pipeline as quickly as possible and to fix bugs; operations wants to keep the lights on and the systems running without downtime, whose likelihood of course is increased when changes are introduced to the production system.

DevOps is no panacea says Sven Gerjets, Senior Vice President for Information Technology at DirecTV. He says, “DevOps is all the rage. If you believe what you read in the tech press, DevOps could even be IT’s saving grace. And predictably, as with most hot IT buzzwords, DevOps is now being used to sell everything from consulting services to cloud services to software.” That statement is obviously cynical.

Mr. Gerjets says it is not enough for operations and development to work more closely together. Rather, the whole IT org chart needs to be structured with this goal in mind.

In what has become cliché, he says the organization needs to break down silos. This is the responsibility of the CIO.

Subsequently, there is a need to measure code and system quality using metrics.

Diego Lo Giudice of Forrester Research says it is important to use metrics that measure value.

He says existing means of measuring development are too focused on measuring the team’s productivity and keeping projects within costs, scope and project dates. A better approach is described as follows: “The truly successful app-dev leaders will be those who focus on delivering constant value and incremental improvement to their business.“

Among the metrics he proposes for measuring value and improvement to the business are:

  • number of features delivered
  • number of defects fixed
  • number of lines of code delivered
  • functionality of function points delivered (“Function point” is a way to quantify business functions provided by the system, so it is a metric all by itself.)
  • number of incidents handled
  • reduction in the number of bugs
  • mean time to repair

These metrics were written for the Agile development organization, but they closely align with the DevOps goal of bringing development and operations teams together, especially “number of incidents handled” and “mean time to repair.” These satisfy both the quality and uptime goals of operations as well as the productivity measurement goal of development teams. Of course the organization can add metrics to the list, including the obvious one: uptime. DevOps can facilitate an agreeable definition.

Using well-defined metrics to measure both sides of the organization is a step towards “breaking down silos” as Mr. Gerjets mentions.