To genuinely measure leadership capability across an organisation, you need real evidence that holds up independently of any one individual's personal opinion, decision quality that can be traced and reviewed, development pathway completion that shows real progression rather than attendance, retention specifically among high performers reporting to each leader, and a structured capability assessment repeated on a consistent cadence. Most businesses substitute a proxy for one of these, usually engagement survey scores, and mistake the proxy for the real thing.
Why Engagement Scores Aren't a Leadership Capability Measure
Engagement scores are, genuinely, useful. They are not a leadership capability measure, and treating them as one is one of the most common and consequential mistakes I see. A team can report high engagement under a leader who avoids every hard conversation, defers every difficult decision, and quietly lets performance issues slide because addressing them would be uncomfortable. Engagement measures how people feel about their week. It doesn't measure whether the leader is actually building capability in the people around them, making sound decisions under pressure, or developing the next generation of leaders beneath them.
The Question Engagement Scores Can't Answer: Would this leader's team still be performing well in a year if the leader stopped actively managing every single detail themselves? Engagement surveys don't ask that question. Real capability measurement has to, directly and repeatedly.
I understand why engagement scores became the default proxy. They're cheap to collect, easy to benchmark against other companies, and produce a single tidy number a board slide can display without much explanation. Genuine capability measurement is harder on every one of those dimensions, which is exactly why so few businesses do it properly even when the CEO privately knows engagement scores aren't answering the question they actually care about. Easy and correct aren't the same thing, and leadership measurement is one of the clearest places I see that gap show up.
The Signals That Actually Measure Leadership Capability
Decision quality is the first of these signals, and genuinely the hardest one to measure well, which is exactly why most businesses skip it. It doesn't mean grading every decision after the fact. It means periodically reviewing a sample of a leader's material decisions against the information available at the time, and asking whether the reasoning held up, not just whether the outcome happened to work out. A leader who makes sound decisions with the information available, even when results are mixed, is demonstrating real capability. A leader who gets lucky outcomes from poor reasoning is a risk the business hasn't yet noticed.
- Decision Quality: Periodic review of a sample of material decisions against the reasoning and information available at the time, not just the outcome.
- Pathway Progression: Real movement through a defined development pathway, not attendance at training, measured by what someone can now actually do.
- Team Retention: Retention specifically among strong performers reporting to each leader, a sharper signal than overall team turnover.
- Structured Assessment: A consistent, repeated capability assessment across the leadership layer, so change over time is measurable rather than anecdotal.
The second signal, real pathway progression, is a good deal easier to measure in practice but just as often, if not more often, measured quite badly. Businesses that do try to track development usually count attendance, workshops completed, hours logged, sessions attended, which measures effort spent, not capability gained. Real pathway progression means being able to point to something concrete a leader can now actually do that they genuinely couldn't do reliably a year ago, a decision category they now own confidently, a team situation they can now navigate without escalating it upward. Attendance is an input. Capability gained is the actual output, and conflating the two is one of the more forgivable but still costly measurement mistakes I see.
Why Team Retention Is a Sharper Signal Than Overall Turnover
Overall team turnover, taken on its own, is a genuinely blunt instrument, because it doesn't distinguish between losing your weakest performers, which can be healthy, and losing your strongest, which is a serious signal about the leader above them. Strong performers generally have options elsewhere. When they leave a specific leader's team at a higher rate than the rest of the business, that's rarely coincidence. It's usually a direct signal about how that leader is developing, or failing to develop, the capable people around them, and it's a far more honest measure than anything a survey captures, because it's revealed by behaviour rather than reported opinion.
I'd add a caveat here, because this signal is genuinely powerful but easy to misuse if applied too bluntly. One strong performer leaving isn't necessarily evidence of anything, people leave for personal reasons, better offers, life changes that have nothing to do with their leader. The signal is the pattern, not the individual event: does this leader's team consistently lose its strongest people at a materially higher rate than comparable teams elsewhere in the business, over multiple quarters, not just once. That repetition across time is what separates a genuine capability signal from noise you'd otherwise mistake for one.
Building a Cadence, Not a One-Off Assessment
A single capability assessment on its own, however carefully and thoroughly designed, tells you where leadership stands today. It doesn't tell you whether it's actually improving, which is usually the question that matters most to a CEO trying to decide whether their investment in leadership development is working. That requires the same structured assessment repeated on a consistent cadence, ideally annually at minimum, so genuine trend data accumulates over time instead of a single snapshot that nobody has anything to compare it against. Businesses that skip the repetition end up with a number that sounds precise and means very little, because precision without a trend line is just a data point, not evidence of anything changing.
I'd also want to flag a specific trap that catches even businesses that genuinely do repeat their assessment properly: quietly changing the instrument between measurement rounds. A capability assessment that measures something slightly different each year, a new framework, a new set of questions, a different vendor, produces numbers that look like a trend but aren't actually comparable to each other. The instrument has to stay consistent for the trend to mean anything, which sounds obvious stated plainly but is genuinely one of the more common mistakes I see once businesses start taking measurement seriously.
- Stop treating engagement scores as a capability proxy — They measure something real and useful, just not leadership capability. Track them separately, not as a substitute.
- Sample decision quality, don't grade every decision — Review a representative sample of material decisions periodically against the reasoning available at the time, not just whether the outcome worked out.
- Track retention specifically among strong performers — Overall turnover is blunt. Losing your best people from a specific leader's team is a sharp, honest signal worth watching closely.
- Repeat the same assessment, on the same cadence, every time — A single snapshot tells you where you stand. Only a consistent, repeated measurement tells you whether capability is actually improving.
There's also a genuine governance dimension worth naming explicitly here, because it's what actually gets this taken seriously at the executive level rather than treated as an HR exercise nobody outside HR pays attention to. When decision quality, pathway progression, and retention data get reviewed by the CEO and the board alongside financial metrics, on the same cadence, leadership capability stops being a soft topic discussed once a year in vague terms and starts being treated with the same rigour as revenue or margin. That shift in how seriously it's treated tends to matter as much as the measurement itself, because metrics that never reach a decision-making table rarely change anyone's behaviour.
What This Looks Like in Practice
Businesses that genuinely measure leadership capability well don't treat it as a separate, dreaded annual event that leaders brace for and then quietly forget about for the following eleven months. It becomes a genuine input into promotion decisions, into where development investment actually goes, and into which leaders get more autonomy versus more support. Done properly, the measurement genuinely isn't punitive at all. It's diagnostic, in the same way a business routinely tracks revenue or margin, not to punish anyone for underperformance after the fact but to genuinely know, honestly and continuously, precisely where the business actually stands so real decisions can be made with real information instead of impression.
None of this genuinely requires an elaborate system or expensive tooling to get started properly. A business can begin with something as simple as a shared spreadsheet tracking pathway progression and a quarterly note on strong-performer retention per leader, reviewed honestly at the leadership team level. The sophistication of the system matters far less than the discipline of actually looking at the same signals, consistently, over time, rather than relying on impression and the loudest recent anecdote to form a judgement about who's genuinely developing capability and who's coasting on a strong team they inherited.
The Distinction That Actually Matters
You genuinely can't improve what you're not actually measuring, and most businesses aren't measuring leadership capability at all, whatever they might believe about their own process, they're measuring how people feel, which is a different, much easier, and considerably less useful thing to track. Genuine decision quality, real pathway progression, honest retention of strong performers, and a genuinely consistently repeated assessment together give you something engagement scores will never quite offer: a genuinely honest, trackable answer to whether leadership capability in your business is actually getting better, staying roughly flat, or quietly eroding while everyone around the table simply assumes it's fine.