Twelve tips for measuring improvement

Understanding what works to improve health care is vital in order to share and spread success and to ensure we learn from our mistakes. These tips will help you embed effective measurement into your project and avoid some of the pitfalls that lie on the route to successful health care improvement.

1. Identify and engage stakeholders from the beginning

Think about who will be influenced by, and can influence, the success of your improvement effort. This will almost always include patients and carers, clinicians and managers. Make sure you find out what success would look like for them. You can’t measure everything that’s important to your stakeholders, but by making informed decisions you avoid a situation where key individuals aren’t convinced by your data.

2. Engage a statistician and information analyst early

Building general skills in measurement across your team is an excellent idea. However, there are subtleties in design, analysis and interpretation that make it important to talk to a statistician or information analyst about your plans at an early stage. They can help you plan what and how to measure, and can advise on study designs.

3. Develop a clear shared aim for the improvement initiative

Once you’ve engaged the right people, work with them to develop an aim statement for your improvement initiative. This should be specific, measureable, attainable, relevant and time-limited (SMART). It should also be as patient-centred as possible, this will help to keep your work focussed.

4. Take time to explore potential solutions, then make it clear how they should work

Carefully review the evidence for the different solutions you could use to improve care, and talk to patients, carers and members of staff about their ideas. Tools such as process mapping can help you understand the existing system in order to maximise your chances of success.

Once you’ve decided on a small number of solutions to test in practice, clearly articulate how these changes will help you achieve your aim – this is your ‘programme theory’ or ’logic model’ and will guide how you implement and evaluate your improvement.

5. Design a set of measures that will help drive your project forward

Design a small set of improvement measures which will help you understand whether the day-to-day work of the initiative is helping to achieve your aim. These should be clearly linked to your aim and available in close to real time.

The data should not be overly burdensome to collect and easy for stakeholders to understand. Where possible use existing measures and routinely collected data, but don’t let this blind you to the data you actually need to make the initiative successful. Your improvement measures will not be the only measures you will need, but they represent a crucial piece of the jigsaw.

6. Use qualitative data as well as quantitative data

Qualitative data can be invaluable to you in gaining a depth of understanding that quantitative measures cannot describe. Using both types of data together is more powerful than either by itself.

7. Write and test ‘operational definitions’ of your chosen measures

In his book Quality Health Care: A Guide to Developing and Using Indicators, Dr Robert Lloyd describes an operational definition as ‘a description, in quantifiable terms, of what to measure and the specific steps needed to measure it consistently’. This is crucially important in ensuring your data is high quality. A good test is to give your definition and the raw data to a colleague and ask them to calculate the measure. If you get different answers there could be an ambiguity in your definition that needs to be fixed. You can also test data collection processes in this way.

8. Create an electronic database for any data not already housed electronically

Talk to someone with expertise in database and spreadsheet design to find out what your options are for storing new data electronically, but be careful not to duplicate existing databases. This database will be useful during the project and essential to monitor long-term sustainability.

9. Collect baseline data, and consider what control groups you may need

In order to clearly demonstrate success you will need good quality baseline data on how things were before your improvement initiative. Depending on the nature of your improvement you may also need to collect data from other settings in order to make comparisons for evaluation. A statistician can advise you on what baseline data you need.

10. Use statistical process control charts to understand progress on a regular basis 

Statistical process control charts are a powerful visual tool, showing your progress on a daily, weekly or monthly basis. They are also statistically rigorous and simple to use, and will enable you to make good decisions based on your data. You should update, review and act on the charts at least on a weekly basis, sometimes daily. This article by Benneyan et al provides a useful introduction and gives some example charts. For more detail see The Health Care Data Guide: Learning from Data for Improvement, by Provost and Murray.

11. Take action according to what the data is telling you

Statistical process control charts will help you decide what action to take at each stage in your improvement work – for example, whether you need to investigate specific events or step back and look more generally at the system. The charts can make it clear whether your changes are working or whether you need to rethink.

12. Think early about embedding measurement into routine work

Throughout your initiative, think about which measures will be available in the long run, especially after any project funding has finished. Also consider any new measures you need to embed for long term success. Remember that your choice of measures may vary over time, so don’t commit to resource-intensive IT systems until you are sure which measures are needed.

Dr Tom Woodcock is a Theme Lead for NIHR CLAHRC Northwest London at Imperial College London and is a Health Foundation Improvement Science Fellow, www.twitter.com/DrTomWoodcock