Ben Goldacre (who seems to be one of this blog’s favorite
sources) tears into the Durham fish oil trial.
to test whether giving omega-3 supplements would improve the GCSE scores of
children. Unfortunately there were clear problems with the trial design. In the
face of criticism the organisations involved refused to give out information on
the experimental setup and even claimed not to be running it as a trial (despite numerous statements to the
press). GCSE scores did not generally increase. Despite this, now positive results are claimed – largely because what is measured has been changed to suit
the data. The most vexing thing about the whole affair is that the
trial could have been done in a proper manner for the same amount of money.
Medical studies are rigorously controlled in order to
safeguard the rights and well-being of the subjects. Studies on children
are particularly sensitive. In the Durham case one could probably complain that
it was not up to the standards of most such studies – did the pupils and parents
really give informed consent? Would they have approved of the study if they
knew how it would be presented in the media? Fortunately the level of risk
seems to have been minimal and clearly the children had a choice in
taking the supplements or not. The people responsible also likely thought they were
doing something helpful. That does not absolve them from blame, but the problem is not an unethical study in itself.
The reason so much effort normally goes into designing scientific
trials is to maximize the amount of information that can be gained about while reducing the risk of being misled by the
myriad of other factors that affect the investigation. Part of this is just
cost-effectiveness, but there is also an ethical underpinning: spreading
misleading or mistaken information impairs science and decision-making. Fraud and bias hurt other research by introducing errors
that can only be corrected by doing even more research showing it to be in error.
But it is very hard to avoid bias. Among many other bias sources, the desire to
see one’s pet theory vindicated often fool even conscientious researchers into subtle
mistakes. Results also tend to be biased towards the interests of the funding
source.
In the case of the Durham trial it is clear from their
statements that the people involved have a bias towards the theory that
supplements are good and (in the case of the company) have a financial interest
in promoting it. The study had serious methodological flaws that would have
been a problem in any case, but the data was interpreted in a way that seems to
bias it towards the desired conclusion. Not only have the results been
presented in the media as a success, but at least one (premature!) reference to the study has
crept into a scientific paper in a renowned journal (Fernando Gómez-Pinilla, Brain foods: the effects
of nutrients on brain function, Nature Reviews Neuroscience 9, 568-578 July
2008) where it may mislead readers.
One can also criticise the study from a marketing ethics perspective.
Ethical marketing involves taking responsibility for products and decisions,
dealing fairly with stakeholders, respecting consumer rights and safeguarding
the interests of vulnerable consumers, such as children. The flip-flopping over
whether it was a trial or not shows a problem with taking responsibility. The
refusal to give information about the study goes against dealing fairly with stakeholders,
who in the case of public health and education clearly involve not just
researchers but the public at large. There is also a possible issue of using
the children of Durham as advertising; while nobody’s privacy seems to have
been infringed some might have reservations against being promoted as arguments
for a particular product – especially when the argument is weak.
To me the greatest annoyance with the whole affair is that
it missed the chance of doing good research on an important question. There are
reasons to think certain fatty acids can have a positive effect on developing
brains. If they have, the social gains of ensuring adequate uptake could be
very large. Yet we have very little solid data. This is something the Durham
study could have helped answer. Now the most likely result will be bringing the
area into disrepute.
One might question whether the close cooperation between a
public institution and a commercial company was appropriate. But the study would
not have been possible without the cooperation, and addresses a question of
public interest. If the study had simply been handled better, perhaps with
outside help to safeguard against bias and to make a proper study, the result could well have been both to
the benefit of the company, the county and the public. Now the opposite was
achieved.
“If the study had simply been handled better, perhaps with outside help to safeguard against bias and to make a proper study, the result could well have been both to the benefit of the company, the county and the public”
There were offers (even from Ben himself, I believe) to help with this trial. I’m afraid I’m more sceptical with regards to the aims and potential achievements – no nutrapill company is going to gamble £1 million of product if there is any doubt in the result and the raft of other publications would have lead them to believe that PR rahter than scientific rigour was the way to go.
The chap in charge (Dave Ford) in my opinion has allowed his position on the council to be abused by a private company, has allowed mass pill-peddling on kids (with no possibility of finding out anything new) and has promoted the idea that pills solve complex social problems.
I’m amazed he still has his job.
If a pharmacutical company wanted to try out some experimental mind-altering drugs (for how else can one describe the claimed effects of concentrated fish oil capsules on brain function) on children there would be outrage.
Yet it’s OK if the pills are “natural” and produced by some nice friendly people from a little cottage industry (worth £billions) that is, of course, only concerned about the health and welfare of our children.
Strange but true.
People are generally far more accepting when something is branded as natural or produced by an “underdog” than they should rationally be. For example, most people are queasy about using a therapy to change gene expression in order to enhance the memory of children, but are quite ready to have mothers supplement with choline or choline-rich food when told it can likely improve children’s cognition – but the two things are exactly the same thing.
Comments are closed.