DORA Metrics: the Right Answer to measuring engineering team performance
“What metrics should I use to measure my engineering team’s performance?”
I get asked this question often enough to be worth an FAQ.
You might expect my answer to be “it depends” but no: I believe there’s a single right answer! More precisely, there’s an established set of metrics that are so good, so widely applicable, that they should be your starting point.
Jacob aka @jacobian
The DORA metrics mentioned in the article – Deployment Frequency (DF), Lead Time For Changes (LT), Change Failure Rate (CFR), and Mean Time To Recovery (MTTR) – are indeed a great starting point for measuring the performance of an engineering team. These metrics have been extensively researched and found to correlate with successful, high-performing teams.
However, it is important to note that these metrics should not be treated as rigid targets from the outset. It is suggested to measure and monitor these metrics for a period of time before deciding what is considered “good” for your specific team. Prematurely setting targets can lead to Goodhart’s Law, where the measure becomes a target and loses its effectiveness as a measure.
Furthermore, it is crucial to carefully define ancillary terms associated with these metrics. Terms such as “failure,” “recovery,” “change,” and others should be clearly defined to ensure consistency in measurement. Take the time to define how these values will be measured within your team.
The article also provides real examples of how different types of teams can apply the DORA metrics. For example, a feature team may measure DF by counting deploys per day, LT by measuring the time from when an issue is moved to “In Progress” to when it is closed, CFR by determining the percentage of deploys causing a failure, and MTTR by calculating the time to resolve incidents or bugs.
Similarly, a security engineering team may measure DF indirectly by looking at the teams they are embedded in, LT by monitoring their own changes, CFR by tracking security issues introduced during their involvement, and MTTR by measuring the time between initial discovery and final remediation.
An SRE team, on the other hand, may monitor DF and CFR for each component team they support and measure MTTR on an organization-wide basis.
Are you an Elite DevOps performer? Find out with the Four Keys Project explores the four key metrics that organizations can use to evaluate their DevOps practices and drive continuous improvement. I found it to be a valuable resource for anyone interested in enhancing their DevOps performance.
Overall, adopting the DORA metrics requires skill and understanding of the subtleties involved. While these metrics provide a solid foundation, it is also essential to consider additional metrics specific to your team’s goals and objectives. Regular monitoring, root cause analysis, and adjustment based on the relative change in performance can contribute to continuous improvement and align team efforts towards success.
