Regulation and quality improvement: A review of the evidence

Vast amounts of resources are expended on regulating healthcare with three key purposes:

• to improve performance and quality
• to provide assurance that minimally acceptable standards are achieved
• to provide accountability both for levels of performance and value for money.

As it is not possible to achieve optimal performance across all three purposes simultaneously, trade-offs are required. This report address the question: when trying to improve health through regulation, what works? 

We regulate in an empirical void, often addressing anecdotes and hysteria with far-reaching initiatives. (Brennan, 1998, p 725)

Vast amounts of resources – financial, organisational and human – are expended on regulating healthcare. Regulation has three key purposes:

• to improve performance and quality
  
• to provide assurance that minimally acceptable standards are achieved
  
• to provide accountability both for levels of performance and value for money.

It is not possible to achieve optimal performance across all three purposes simultaneously and so trade-offs are required. The Health Foundation’s Quality Enhancing Interventions (QEI) project undertook a literature review to address the following question: when trying to improve health through regulation, what works?

Findings

Overall, the evidence available to answer this question is sparse. Research evidence about the impact of regulatory interventions on quality of healthcare is drawn primarily from observational studies. This means that the links between regulation and improvements in quality are primarily associative rather than causal. The evidence base largely emanates from the US, making the contextual interpretation challenging for other countries. We have reviewed what evidence does exist under the following headings: institutional regulation, professional regulation and market regulation.

Institutional regulation

There are two main types of institutional regulation:

• those concerned with direction, that is, defining and communicating expected levels of performance
• those concerned with surveillance and enforcement, often referred to as ‘external oversight’.

Evidence in brief

Target setting

Four studies (Bevan & Hood, 2006;Alvarez-Rosete et al, 2005; Auditor Gen for Wales, 2005; Bevan & Robinson, 2004) drew on routinely available performance data and concluded that targets have been associated in England with reduced waiting times for:

• inpatient care
  
• urgent ambulance calls
• access to accident and emergency treatment.

Targets have been widely used in public health programmes such as Health for All (WHO, international); Health of the Nation (Department of Health, UK); and Healthy People 2000 (Centers for Disease Control, USA). Case study findings and routine data suggest that these sets have had mixed success (Department of Health, 1998).

Standard setting

Four independent studies found an association between standards – as laid out in the Coronary Heart Disease National Service Framework in England (Department of Health, 2000) – and improvements in quality (Graham et al, 2006; Ramsay et al, 2006; Ramsay et al, 2005; Hippisley-Cox et al, 2005). Progress reports published by government also provide data on generalised improvements in coronary heart disease care (Department of Health, 2005).

Six studies (Sheldon et al, 2004; Abacus International, 2004; National Cancer Director, 2004; Wathen and Dean, 2004; Bloor et al, 2003; Hassan et al, 2005) examined the impact of the National Institute for Health and Clinical Excellence (NICE), a quasi-regulatory body. Overall, these studies found that NICE guidance had a mixed impact on patient care, and was most effective when supported by other levers for change.

Accreditation

Within the US healthcare system (which relies heavily on accreditation), one multistate comparative study found that accredited organisations provided higher quality care for cardiac patients (Chen et al, 2003).

Two studies found a disjuncture between accreditation scores and alternative measures of performance, quality and safety (Miller et al, 2005; Griffith et al, 2002).

One study of US health plans found that accredited plans had significantly higher quality of care scores across seven out of nine measures (Beaulieu & Epstein, 2002).

One randomised controlled trial conducted in South Africa found improved compliance with standards following the introduction of an accreditation scheme, but little evidence of improvement in quality indicators (Salmon et al, 2003).

Inspection

Three studies (two in the UK and one in the US) found that inspection, or the prospect of formal inspection, was a catalyst for improvement on the part of regulated organisations (Day and Klein, 2004; Benson et al, 2004; Office of Inspector General, 1999).

Professional regulation

Licensure is probably the most widely used intervention in professional regulation. Most countries have a system for controlling entry into the medical professions and an official register of physicians who are licensed to practise. In a growing number of countries, licensing or registration extends to other healthcare professionals such as nurses. The Institute of Medicine (2000) publication, To Err Is Human, recommended that professional licensing bodies consider continuing qualifications over a lifetime of practice, not just at initial licensure or registration. Despite the international prevalence of physician licensure, there is little evidence available about its impact on quality of care.

Evidence in brief

In the US, certification and recertification are processes that enable physicians to demonstrate achievements and competencies that are beyond the minimum standards required for licensure. We found one systematic review (covering literature published from 1966 to 1999) (Sharp et al, 2002) and four subsequent studies that indicate a positive association between specialty physician certification and higher quality care (Chen et al, 2006; Silber et al, 2002; Prystowsky et al, 2002; Norcini et al, 2002, 2001, 2000). However, evidence is not available to prove that certification has secured improved quality.

Market regulation

Healthcare markets generally require regulatory interventions for the following reasons:

• manage competition: to ensure a ‘level playing field’ and allow market forces to deliver efficiency by limiting concentration of power in monopolies or cartels
• protect patients: to assess quality of services, ensure that robust grievance and appeals procedures are in place, protect confidential patient data and ensure financial solvency controls
• provide public accountability: to ensure transparency of performance and information so that consumers can make decisions based on cost, outcomes and quality
• manage supply: to manage conditions of participation and influence the availability of care (addressing both oversupply and undersupply).

Evidence in brief

Managing competition

One US multistate study (Christianson et al, 1997) covering health maintenance organisation (HMO) mergers from 1985 to 1993 found that regulatory interventions seeking to limit mergers and acquisitions did dampen market consolidation, but that states with stronger anti-takeover regulation had higher HMO failure rates. The regulations may have prevented non-viable units from merging and this could have contributed to their demise, with implications for access to care.

We found nine US studies that examined the impact of conversions from non-profit or public hospitals to for-profit status (Leone et al , 2005; Shen, 2003; Picone et al, 2002; Sloan, 2002; Thorpe et al, 2000; Desai et al, 2000; Young and Desai, 1999; Mark, 1999; Needleman et al, 1999). Regulation can dampen conversion activity; however, none of the studies reported on the role that regulation has played in limiting the undesirable consequences of conversions. Four studies examined provision of uncompensated care for low-income and uninsured patients following hospital conversion. Three of the four found that hospitals that had converted from public to for-profit status provided lower volumes of uncompensated care (Thorpe et al, 2000; Desai et al, 2000; Needleman et al, 1999); the fourth found no impact (Young & Desai, 1999). One study found an association between conversion to for-profit status and higher mortality rates (Picone et al, 2002), while another found an association between conversion and higher rates of pneumonia complications (Sloan, 2002).

Accountability

Of four studies on cardiac surgery in New York, three found that mandatory performance reporting had a positive effect on patient care (Cutler et al, 2004; Petersen et al, 1998; Hannan, et al, 1994) one did not (Dranove, et al 2003). Survey data suggest that mandatory reporting and public disclosure can have unintended consequences, for example, discouraging treatment of high-risk patients (Narins et al, 2005; Weissman et al, 2005).

Managing supply

In the USA, certificate of need (CON) is a permit issued by a governmental body to an individual or organisation proposing to construct, modify, or close a health facility; acquire major new medical equipment; modify a health facility; or offer a new or different health service or discontinue a service. CON regulation as applied to general hospital services has not been successful in curtailing overall expenditure. In respect to impact on quality, one study from the 1980s found that more stringent CON regulations were associated with higher mortality rates across a range of clinical conditions (Shortell and Hughes, 1988). A more recent multistate study found that patients with acute myocardial infarction in states with CON were less likely to be admitted to hospitals with revascularisation services than patients in states without CON; however, there was no difference in mortality rates (Popescu et al, 2006). As a tool to ensure minimum volumes of cardiac surgery, CON has been associated with lower mortality rates (Vaughan-Sarrazin, et al 2002).

Rate-setting refers to a process of controlling prices, where regulators prospectively define a maximum amount that providers are allowed to charge their customers. Rate setting has been the subject of a number of multistate studies in the US, with mixed findings. In two studies rate regulation did not have an adverse effect on patient mortality or population mortality, despite lower admission rates (Smith et al, 1993; Sloan et al, 1986) . However, another study found a statistically significant association between mortality rates and the stringency of rate regulation (Shortell and Hughes, 1988).

In a time series analysis examining outcomes from 1974 to 1983, mortality rates decreased more slowly in states with rate-setting (Gaumer et al, 1989).

Data from New Jersey in 1990-96, indicated that the discontinuation of rate-setting in favour of a price competition model was associated with an increase in relative mortality rates for a number of conditions (Volpp et al, 2005).

In terms of impact on the diffusion of innovations, rate-setting does not appear to hamper the overall availability of cost-increasing technologies, but can temper excessive use and duplication of services (Sloan et al, 1986b; Romeo et al, 1984).

Formularies are lists of preferred drug products that limit the number of options available within a therapeutic class for purposes of drug purchasing, dispensing and/or reimbursement. Drug formularies’ influence on quality of care has not been researched robustly. A literature review noted a deleterious effect of formularies on patient outcomes (Lexchin, 2002). However, it also highlighted methodological limitations which bring the results into question. According to a national survey, the formulary introduced by the Department of Veterans Affairs in the US was not perceived by physicians to adversely affect patient care (Glassman et al, 2001).

About this study

We conducted systematic searches of a range of electronic databases: Medline, the King’s Fund, Database of Abstracts of Reviews of Effects (DARE), Cochrane Database of Systematic Reviews, Organisation for Economic Co-operation and Development, World Health Organization, Agency for Healthcare Research and Quality and the Commonwealth Fund.

We searched for empirical evidence about the effects of various approaches to regulation on quality of care. The review included different types of research designs: systematic reviews, randomised controlled trials and quasi-experimental and observational studies. We adopted broad inclusion criteria because of the methodological challenges inherent in assessing complex organisational and managerial interventions such as regulation.