Part 5. Create Clear Measures of Success

Too often, I see scorecards where the metrics point perpetually “up and to the right.” Often that’s because the metric tracks activities rather than actual progress toward goals. It’s easy to measure the number of features we added to an application or the number of routers we upgrade, for example. What we should be asking and measuring is what goal we are trying to achieve for stakeholders and how close we are to achieving that goal. A program or application can function perfectly, but if it doesn’t help the business accomplish its goals, it’s pointless. For example, with the newly signed Sergeant First Class Heath Robinson Honoring our Promise to Address Comprehensive Toxics (PACT) Act, we strive to speed up the rate of claims resolution by automating more of the process. So, we can measure our success by tracking the percentage of claims automated or the time to adjudicate a claim, both of which will tie fairly directly to the success of our benefits organization. What we shouldn’t measure our success by is the number of claims automated — that’s a number that’ll always go “up and to the right.” We should always think analytically about what we are measuring and convince ourselves that it is the closest proxy for accomplishing the goals of the organization.

Financial measurements are also important to define and track. In addition to ensuring that resources are allocated based on business priorities, we can define “cost per” measurements that track whether we are being effective in how we use those critical resources. For instance, we can measure the cost of desktop support per desktop. As we increase our employee base, we should expect this cost per desktop metric to decrease as we become more efficient at delivering support. While this cost can increase due to tech inflation and the rising cost of the components, the industry trend has been in the downward direction, as we use automation to both decrease the cost of support and improve the support experience. As a team, we should define financial metrics that can accurately represent our per-unit cost in delivering a service, and we should create goals for improving those metrics over time. We are just in the formative stages of developing such metrics at the VA.

In the Office of Information and Technology, we’ve defined a set of metrics across each area and embody these in a set of organizational OKRs (Objectives and Key Results).  Each team has their own set of OKRs, and we elevate a finite set to be org-wide OKRs.  We set tough but achievable near-term goals for these, review them regularly, and adjust the set periodically.