Measuring success

How do you know how healthy you are? Do you think about how many calories you’ve had today, or do you look at your diet over the longer term, how much you exercise and your general wellbeing? Individually these factors aren’t that useful – but together they paint a picture of your health.

The same difficulty exists when it comes to measuring success in the NHS.

The Health Foundation’s Constructive comfort: accelerating change in the NHS recommends moving away from understanding performance through targets to focus on a broader and more balanced assessment of quality.

The selection of vanguard sites for the New Care Models Programme and CQC’s ‘Quality of Care in a Place’ pilot signal an opportunity for a fresh approach, and an opportunity to redefine what success looks like for the NHS.

A flaw in the system

Let’s look at how success is currently measured in the NHS using two examples – Nottingham University Hospitals NHS Trust and Basildon and Thurrock University Hospitals NHS Foundation Trust. 

Against current performance metrics the success of both hospitals is debatable. For example, neither hospital performed well against the 95% threshold for the four hour A&E waiting time target this winter. For the national two-week cancer waiting time target Nottingham has failed to meet the 93% threshold for three consecutive quarters. And despite the Secretary of State singling out Clare Panniker as an excellent Chief Executive, the financial viability of the trust was called into question in 2014 when the financial loss of the trust was predicted to be bigger than expected.

But is this the full picture?

Not only is Nottingham NHS Trust part of an integrated primary and acute care systems vanguard site, it is also succeeding in a different area – by becoming renowned for its innovative quality improvement programmes. For example, Nottingham’s ‘Better for You’ programme is a whole hospital initiative. The programme empowers staff to make change happen in their own areas of work by giving them the freedom to put forward their own ideas for improvement through the ‘Just Do It’ scheme. Staff engagement, tick. Capability for change, tick.

Basildon and Thurrock has been commended for its good management practices, and for creating an enabling environment that gave the new leadership team the headroom it needed to make improvements. A firmer approach to underperformance has also been implemented, and its maternity unit became the first in the country to get an outstanding grade. Staff engagement, tick. Capability for change, tick.

Both hospitals have their pros and cons, but the narrowness of the current metrics overlooks some important areas of success. 

What would change look like?

Constructive comfort suggests using a more holistic approach to define success and measure improvement. This includes taking into account six key factors for measuring success (Figure 1), crucially with equal parity given to the different factors.

Figure 1.

Examples of measuring these factors in the current landscape are limited. For example, how is the local health economy being measured? The Five Year Forward View success regime emphasised the need for the transformation of local health economies, but the details of how to achieve this have yet to be revealed. Factors like patient flow across pathways and financial balance across local health economies also need to be considered.

How are long-term outcomes being measured? A variety of monthly, quarterly and annual data on various measures is provided by NHS England, but are conclusions being drawn in the right way? Take for example a patient admitted for surgery because of a hip fracture. The outcome of this surgery is not usefully recorded over the longer term. For example, did the operation lead to more surgery? Did the patient require additional services such as physiotherapy?

The long and the short of it

Success in health care is a difficult measure because it depends so largely on human nature – things like trust and building relationships are marks of success but difficult to define or quantify.

Perhaps those trusts deemed less successful by current measurement standards are incubating their potential to become the high performers of the future, by focusing on longer term, innovative improvement measures. We need to change the way in which success in the NHS is perceived to improve quality and efficiency at the pace and scale needed, and move towards new models of care.

If we accept that our current culture of favouring goals that deliver immediate gratification cannot be sustainable for the NHS, why haven’t changes been made? Part of this is the political challenge: as shown in a 2001 Massachusetts Institute of Technology paper, creating and sustaining improvement almost inevitably generates a dip in short-term performance. So it becomes a question of courage – are we prepared to accept a short-term dip in performance in order to achieve long-term success, viability and enhanced care?

Anna is a Policy Intern at the Health Foundation.