A new performance assessment framework for the NHS

NHS England has released a draft framework for rating the performance of NHS organisations, with the final version due at the end of June. It comes as the approach to overseeing NHS performance in England – and the organisations responsible for it – is in flux. 

The health secretary, Wes Streeting, says he wants to ‘hold providers properly to account’ for performance through new league tables for NHS trusts. Penny Dash’s review of the Care Quality Commission reported ‘significant failings’ and plans for the regulator to assess integrated care systems look unlikely to happen following another pause to the process. The draft NHS England framework confirms overseeing the performance of NHS providers will rest with national teams (or their regional arms), rather than integrated care boards (ICBs). But beyond this, what does the new approach look like and what will it mean for NHS leaders?

The proposals in a nutshell 

An overall ‘delivery’ score will be calculated for each NHS provider and ICB, based on an aggregation of scores across a series of metrics (covering four domains: priorities from the latest planning guidance; finance and productivity; public health and patient outcomes; quality and inequalities). This score (and a ‘diagnostic’ for those at the bottom of the pack) will be used to assign each organisation to one of five ‘segments’ reflecting their overall performance. 

How performance in different parts of the assessment will be weighted to form the overall score is as yet unclear. But there is a proposed ‘override’ for financial performance, meaning organisations in deficit or scoring poorly on the finance domain will have their overall score limited to segment three (described as ‘off-track in a range of areas’). With deficits now widespread, this could limit the rating of many organisations. ICB scores will be adjusted based on the performance of the system as a whole (as judged by an additional set of system-level metrics), while providers will be rated solely on their own performance – drawing criticism from system leaders for potentially undermining integration and system working.

The segment assigned determines the ‘support’ or ‘enforcement action’ from national policymakers. In other words, a mix of carrots and sticks to try and drive up performance. Top performers will receive additional ‘freedoms’. But there isn’t much detail on these yet, beyond proposed additional flexibilities around the use of capital, an opportunity to ‘shape and drive national policy’ and test new models of care. Those deemed underperforming may be subject to the removal of board members, a transfer of functions to other ICBs, placed in the ‘recovery support programme’, and more. The framework signals that ‘greater incentives for performance’ are to come. Work is also underway to assess leadership capability, and a public-facing website will be launched to ‘support transparency’. 

What’s the aim? 

The right approach to measuring performance depends on the overall aims of measurement – for example, whether measuring for judgement, improvement, transparency, or some other objective. Without further clarity, this new approach risks meeting none of them well. 

Measures to support local systems to improve performance might be best seen as a ‘tin opener’ for local leaders to identify priority areas for action, rather than a definitive measure of performance. To target improvement, leaders will likely want disaggregated information on processes and outcomes at different levels of the system. Information to hold ICSs to account for performance (ie for judgement) should act more like a ‘dial’, providing a clear measure of performance where there is little or no doubt about attribution, and where local leaders have a clear role in shaping performance. Measures to inform the public on quality of care or support patient choice (ie measurement for transparency) would likely require simpler information at a less granular level. 

Some metrics may suit multiple purposes, many will not. Healthy life expectancy, for example, is largely determined by factors outside the NHS’s control. While ICBs can and should contribute to improvements in healthy life expectancy over the long-term, this is not a reliable indicator to judge ICB performance in the short term – particularly if the framework is intended to be used as a tool to guide performance management of the NHS by national leaders.

One option, recommended by an independent review for the Department of Health in 2015, would be to develop an approach that combines information at multiple levels: a small set of headline indicators meaningful to the public on the overall performance of the local system; a broader set of indicators for national policymakers on how providers and ICSs are delivering on key national priorities; and a more comprehensive, granular set of indicators for local systems to support improvement. 

Avoiding simplistic aggregate ratings 

Targets and systems of performance measurement can contribute to better performance in selected areas and – overall – the aim of measuring performance across local health systems is a good one. But there are also risks and unintended consequences, particularly when it comes to aggregate ratings. Rating systems as ‘high’ or ‘low’ performing may be superficially clear and easy to understand and compare. But ratings based on composite scores that combine varied metrics can face multiple issues. They can conceal the more complex reality of health system performance and may have harmful effects, potentially shifting the policy debate towards spurious rankings rather than actual (changes in) performance. 

They are also complex to design, requiring judgements about how to weight indicators covering a wide range of diverse areas. And how to reliably track performance over time when indicators that underpin the ratings change and evolve to reflect emerging priorities and new datasets. Then there are unintended consequences, like gaming or distorting local priorities, discrediting the ratings and undermining their impact, or the potential for worsening staff morale. Summary ratings given the complexity of ICSs are unlikely to be a meaningful representation of performance to the NHS or the public. 

Strengthening capabilities to support improvement – the missing link?

Setting the complexities of measurement aside, historically the NHS has over-relied on top-down performance management, with too little attention given to the capabilities needed to support improvement. Any performance measurement or league table system needs to be accompanied by other policies to support organisations to improve in response to the data – for example, building on work in connection with NHS Impact to strengthen improvement capability and align it with the new performance framework. 

The draft framework acknowledges that implementing it ‘may increase the demand for improvement capacity’. But with ICBs and providers facing substantial cuts this year, it’s not clear whether leaders will have the bandwidth or the resources to respond.