KPI-Driven Improvement

How do you know improvement work is making a difference?

In Measures of Success: React Less, Lead Better, Improve More, Mark Graban distinguishes meeting a target, improving performance, and knowing how to improve. These are different questions. A KPI review should help a team understand the system producing the results and choose its next action.

1. Separate activity, process performance, outcomes, and impact

• Activity: ideas submitted, tests started, or projects completed. These show participation and work in progress.
• Process performance: measures such as handoff delays, rework, or first-pass quality that help teams understand how work is performed.
• Outcomes: results customers and the organization need, such as reliable delivery, shorter waits, or fewer defects.
• Improvement impact: the benefit attributed to a change, with its evidence, assumptions, and validation made clear.

The categories can overlap depending on scope. A shorter response time may be an outcome for one team and a process measure in a broader customer journey. Explain what each measure means in your setting. Participation matters, but more submissions do not prove better operational results.

2. Agree on an evaluation plan before the change

Define the problem, the expected effect, and how the team will evaluate it. Specify the measure, its data source, the period being reviewed, and who will maintain it. Use consistent definitions before and after the change, and note relevant changes in volume, work mix, or measurement.

Include balancing measures. A manufacturing team seeking faster output should also watch quality and safety. A healthcare or service team reducing waits should consider reliability, repeat work, and employee workload. A local gain may create a cost elsewhere.

3. Study performance over time

A two-point before-and-after comparison leaves important questions unanswered. Was the earlier point unusually poor? Is the later point within the usual variation? Did another change happen at the same time?

Plot the history and identify when changes were introduced. A time-series chart provides context; a green target indicator alone does not establish a meaningful improvement. Process behavior charts can help distinguish routine variation from signals of change when deeper analysis is needed. This page focuses on the management decisions, rather than chart calculations or detection rules.

Routine variation calls for work on the underlying system, rather than a new explanation for every fluctuation. A credible signal of change gives the team a reason to investigate what changed and whether the improvement can be sustained.

4. Distinguish estimated benefits from demonstrated results

Record whether a benefit is anticipated or observed, how it was calculated, and who reviewed it. Separate time released for other work from an actual reduction in spending. Avoid double-counting overlapping projects or adding an annual projection to benefits already counted for the same period.

A change followed by better performance is evidence to examine, rather than automatic proof of causation. Combine the data with knowledge of the process, the timing of the test, and other plausible explanations. Say when the result remains uncertain.

An illustrative example: faster order fulfillment

A team tests a revised picking layout to reduce order lead time. Before the test, it reviews the historical lead-time data and defines what improvement it expects. It also follows picking errors and employee safety concerns so speed does not come at their expense.

After the test, the team reviews results over time, checks whether order volume or mix changed, and decides whether to adapt or extend the test. It distinguishes any estimated labor benefit from actual spending changes. This is a suggested evaluation approach, not a reported KaiNexus customer result.

Use the review to choose the next action

If the evidence is unclear, decide what information or additional test would help. If the change appears useful, consider how to sustain it and test under other conditions. If it did not help, record the learning and adjust the approach. Closing a project should not prevent later review of its effects.

For the broader choice of balanced priorities, see balanced scorecards and SQDCM. For testing a proposed change, see the PDSA cycle.

Further reading: Mark Graban, Measures of Success: React Less, Lead Better, Improve More, especially the discussion of targets, variation, and evaluating improvement.