Dominic Wilkinson @NeonatalEthics
In the news this morning, the NHS has released data on individual surgeons’ performance, so called “surgeon report cards”. This represents the latest move towards increased transparency and accountability in the National Health Service. Elsewhere in the media today, there are numerous reports of the UK couple who were apparently charged £100 after posting a negative hotel review on an online website.
These parallel stories highlight one concern about certain types of health accountability: sensitivity to the negative impact of reviews (or poor performance figures) could lead to harmful changes in behaviour. For surgeon report cards, one frequently cited concern is that publishing report cards could lead surgeons to avoid high-risk cases. If surgeons choose patients with lower risk of dying, they will potentially end up with a better report card. However, then the results would be misleading (it would be the equivalent of someone getting a higher mark by choosing to sit an easier test). More worrying, it may mean that some high-risk patients are unable to access surgery.
Should we be worried about the negative effect of report cards on surgeons behaviour?
One reason to be sceptical about this concern is that there are some ways of adjusting performance figures to take account of patients’ risk. Websites like MyNHS publish risk-adjusted mortality rates. This adjusts (at least to some degree) a surgeon’s performance results based on how high risk their patients were. It is a bit like the way that Olympic divers’ scores are adjusted for the degree of difficulty of their dive. Choosing an easy dive is not an effective way of gaining a gold medal. Choosing low risk patients will not necessarily improve a surgeon’s overall performance score.
A second reason is that empirical evidence does not show clear evidence that surgeon report cards lead to risk avoidance. New York State was one of the first places to introduce report cards for heart surgery. After these were introduced there were reports of increased rates of referring high-risk patients to hospitals in other US states as well as some cardiologists having difficulty finding a surgeon for their high risk patients. However, a number of other studies, including this one from the UK show no overall risk avoidance. Over time, there had been an increase (rather than a decrease) in the number and proportion of high risk patients undergoing surgery.
However, even if report cards were to lead to some surgeons avoiding risky patients, the question would be whether this harm outweighs the benefits of report cards. One of the major ethical arguments behind report cards is a belief that this sort of transparency is linked to improvement in the quality of care. The other way that surgeons can improve their performance is by careful attention to avoiding complications and improving patients’ outcome. (The analogous way for hotels to improve their ratings on online websites is by good customer service and facilities). Although this appears a plausible outcome of surgeon report cards, there is mixed and inconsistent evidence. A systematic review, published in 2008 concluded that “publicly releasing performance data stimulates quality improvement activity at the hospital level. The effect of public reporting on effectiveness, safety, and patient-centeredness remains uncertain.” One reason for the inconsistent evidence is that data about published surgeon performance are not always of high quality or easy for patients to understand.
Will surgeon report cards improve overall quality of care in the NHS? It is too early to say. However, concerns have been raised about the superficial and limited information published in the MyNHS website. In the data available to date there appear to be no significant differences between individual surgeons, almost all were performing within the ‘expected range’ for their specialty. If this result means that UK surgeons are all already performing at a high level, then it would be very encouraging (though it would also suggest that report cards would be unlikely to improve quality of care). However, it seems more likely that the currently published indicators are not sensitive enough to important differences in outcomes between surgeons. (The reports may be good, but it would be like the teacher who gives every student an ‘A grade’.)
Transparency is important in healthcare, however, it is vital that the correct information is collected. Otherwise report cards risk wasting precious time and resources in unhelpful bureaucracy.
Further reading:
Steve Clarke and Justin Oakley Informed Consent and surgeon’s performance. Journal of Medicine and Philosophy 2004
Nice post Dom and excellent choice of further reading! One point that often gets missed in these debates is this: opponents of the use of report cards often claim that report cards lead to ‘defensive surgery,’ and as you note there is a lack of evidence to substantiate this conclusion. But even if it can be substantiated, this may not be enough to show that any harm has occurred. Some risk-taking surgeons presumably should be more defensive than they currently are and to the extent that report cards lead these surgeons to be more defensive, report cards have led to a benefit, not a harm.
Thanks Steve,
you are right that risk aversion could lead to high risk patients receiving treatment from a smaller group of surgeons who have the appropriate skills to treat them. This might be a good thing. However, the concern is that some high-risk patients will be told that their condition is ‘inoperable’, when in fact surgery would be possible (though high risk). As noted, there is no clear evidence to support an overall reduction in surgery for high risk patients (following introduction of report cards). It may be that the UK in particular this concern is less of a problem because patients have limited ability to choose between surgeons. Here the TripAdvisor analogy falls down. One reason for hotels being paranoid about any negative reviews is because of the intense competition for business, and the strong impact of online ratings (particularly negative ones). There isn’t strong competition between surgeons in a public health care system like the NHS, correspondingly small differences in performance between surgeons have no impact on livelihood.
Comments are closed.