Measuring quality: a complicated task

Why measure quality?

Understanding the best ways to measure and monitor the quality of healthcare is at the core of our improvement work here at the Health Foundation. Within all health organisations, collecting data around quality and safety is also vital to flagging up problems, highlighting issues such as poor patient experience or high infection rates. Not collecting or using this information has resulted in some notable failures in care – Mid Staffordshire, for example – demonstrating just how important it is to monitor quality and respond quickly as issues are identified.

Challenges to measuring quality

Perhaps a harder question to answer is: how should we measure quality of care? We know what quality care should look like. It needs to be safe, effective, person-centred, timely, efficient and equitable. This sounds obvious, but in practice each of these areas is very complex and presents its own challenges when it comes to measurement.

Here we outline some of the main reasons why measuring the quality of care often proves so complicated.

Healthcare systems are not static

Every patient is different and the environment is forever changing. This can make it hard to isolate particular outcomes and measure their effectiveness. The indicators we use to measure quality can be affected by a myriad of other factors, such as the type of treatment someone is receiving, how healthy they are, their medical history, and what quality of care means to them as a patient. All these factors can affect the measurement data.

We also have to take into account other changes to the healthcare environment that are happening beyond the confines of the treatment or intervention but which still affect care outcomes, including changes to organisational structure or staffing. We can try to take these factors into account when measuring quality but it’s often hard to define exactly how the changes have impacted.

Quality means different things to different people

Health Foundation Governor Margaret Goose makes a strong case for including patient and family views when measuring quality. She explains how patients and the wider public have a significant role to play: not only in designing improvements, but in monitoring whether they have the desired impact – not least because they are the only people who really experience the patient pathway from start to finish.

It’s important to note that patients often define quality differently from clinicians and managers. What they view as the problem or value within a system may be surprisingly different from the accepted organisational view. For this reason it’s vital we use patient experience as a core measure of quality, although it can be a challenge to keep staff thinking about this when designing improvement work. And finding ways to respond adequately to patient views is also difficult, when every individual patient will experience quality in a different way.

We shouldn’t assume clinicians have data expertise

A lack of data collection expertise among clinicians being asked to gather or self-report data presents challenges for accurate measurement. Often a competence in data collection is assumed rather than taught, resulting in a difference in interpretation of measures or levels of reporting.

Professor Peter Griffiths from the University of Southampton touched on this when we spoke to him this month about measuring the quality of nursing. He suggests there is still work to be done to ensure we collect and interpret data correctly, even when dealing with seemingly straightforward measures like the incidence of pressure sores. Risk adjustment is also needed when we are interpreting results, he says, to adjust for different patient groups and their relative vulnerability.

It’s not always clear what we should measure

When measuring safety in healthcare, we have sometimes struggled to understand what we should be looking at. For example, should we focus on the absence of safety (by monitoring incidence of error and harm) or on the positive achievement of keeping patients safe?

Learning from other high risk industries encourages a move away from just measuring error as it occurs, towards monitoring elements of the health system or care pathway in an attempt to prevent error occurring in the first place. But this is no easy feat in healthcare, due to the dynamic nature of the system.

We need greater clarity and less variability

Recent research commissioned by the Health Foundation highlighted how important it is to ensure data collection systems are designed and operated to produce reliable indicators of quality. The ‘Lining up’ research, led by Professor Mary Dixon-Woods from Leicester University, found huge variability in how hospitals collected, recorded and reported their rates of central line infections. It concluded that this data wasn’t comparable across hospitals, and may not be a suitable performance measure.

The Health Foundation has commissioned Professor Charles Vincent from Imperial College to lead an important research project into measuring safety, with findings due to be published next year. A key theme emerging from the research is that our understanding of how to measure safety in healthcare remains confusing and unclear; the researchers are proposing a framework to measure and monitor safety in healthcare to enable greater clarity.