Publication bias: what they’re not telling doctors…
So, what exactly is publication bias? The general public don’t know, politicians can’t offer a straight answer – yet the issue strikes to the heart of medicine itself. Publication bias is a problem that cannot be sound-bitten, cannot be summed-up in just a few words, and one that cannot, and should not, be ignored any longer.
We like to be aware of as much information as possible before making decisions; we balance options against each other to gauge what’s best. Naturally, we expect our healthcare professionals do the same. In fact, we expect their decisions be both better informed and better evaluated than our own – so what if information were being withheld from our healthcare professionals? What if some of their decisions were forcibly biased, and ultimately dangerous? Drug prescription is enormously affected by data available in academic literature – before prescribing, doctors need to know as much information about a drug’s safety, efficiency, and efficacy as possible. Current academic literature can render crucial data almost impossible to find – this problem is called ‘publication bias’, and it is compromising the decision-making of professionals whom we place our ultimate trust in.
More accurately, publication bias refers to data from a particular area of research that fails to accurately represent the full range of data collected. It can be thought of as ‘cherry-picking’ data in order to prove a particular result. It is well known that there is a tendency in scientific journals to shy away from publishing negative data: studies show that papers boasting positive results are twice as likely to be published as those with negative results. Not only is this is a damning indictment of a process behind which drug companies both fund, and conduct, their own trials on their own drugs, but in medicine it could be disastrous. Publishing papers which solely show the positive trials of a drug, whilst disregarding the negative ones, means that the very pharmacological literature available is grossly biased. The problem extends further still: poorly designed trials, conducted on patients who are unrepresentative of the target population, and analysed using ineffective and out-dated methods, are used to produce results which grossly exaggerate the benefits of the drugs in question. Some researchers aren’t just omitting negative results, but they are using malpractise to engineer trials with falsely positive results.
Why is this issue coming to a head now? Publication bias has affected scientific research for years, but it is only within the last few that comprehensive reviews and meta-analyses have been widely conducted on specific areas of research – newly finding papers available on certain drugs to be unrepresentative of the overall data.
In Ben Goldacre’s new book, Bad Pharma, the need for more stringent controls on the publication of pharmaceutical research is explored. Goldacre presents the results of a survey by Erick Turner, who, when investigating the results of all anti-depressant trials submitted to the United States food and drug administration, found 38 studies which produced positive results… Alongside a whopping 36 with negative results. However, the most eye-opening fact was that out of the 36 negative trials, only three had their results published – while all but one of the 38 positive studies were made available. Such biased reporting of a drug’s effects is arguably just as useless as conducting no research on the drug at all.
The problem of publication bias is most effectively communicated in Natalie McGauranand’s 2010 review of the issue where she explores examples such as Gabapentin, used to treat neupathic/nociceptic pain and migraines. From nine trials for neupathic pain, seven had negative results, four of which were not published, and the three negative results published were presented with significant spin. When research on effects of Gabapetin on nociceptic pain and migraines was undertaken, there was no evidence to suggest it was any better than a placebo – yet it is still used to treat migraines even today. McGauranand moves on to present examples of bias in the available data for other drugs, such as those relating to Alzheimer’s, ADHD, and the now infamous Class-1 anti-arrhythmics (disastrously used to suppress abnormal heart rhythms in the US, resulting in deaths of more than 100,000 people). While these are extreme examples, similar results were confirmed in John Loannidis’ 2005 paper – he investigated the prevalence of publication bias, ultimately finding it in studies focusing on everything from genetic disease markers to basic clinical research.
One particular drug which illustrates just how desperate our publication problem is is Reboxetine – used as an anti-depressant, and still often prescribed in 2012. Seven initial trials were conducted on Reboxetine, comparing its effect to that of a placebo sugar pill. Out of these seven trials, only one was published – the one result suggesting that Reboxetine’s efficacy is higher than placebo. A 2010 review by the German Cancer Society (GCS) showed, through a meta-analysis, that out of the 13 trials conducted on Reboxetine 74% of data went unpublished. The GCS went on to show that the drug is not only inferior to other SSRI antidepressants, but is also inferior to placebo in terms of harm outcomes. Through publication bias, it was shown that the positive effects of Reboxetine were overestimated by 95%-115% compared to placebo, and around 20% compared to other SSRIs. For medical practitioners to make an informed decision on the administration of any given drug, they need access to all the relevant information. Publication bias, such as the gross exaggeration of Reboxetine’s efficacy, is therefore directly infringing upon our healthcare professionals’ decision-making.
Now that we’ve presented you with the problem, what’s the solution? In fact, there is no short answer. A number of attempts have been made to improve the availability of all research, both positively and negatively resultant, such as the formal registering of all trials before any research is carried out. Solutions, however, inevitably carry new problems: for instance, formal registration laws have led to many companies simply not registering their trials before conducting them, and being permitted to publish their results regardless. Another FDA law attempted to force researchers to publish within one year of conducting trials, but, again, a review published in the BMJ in January 2012 found that only one in five actually complied.
To make some real headway, we need both strong, political motivation and a firm direction. We need to ensure that all drug trials are published, including those concluding negatively, and that literature concerning all drugs currently in use is updated regularly and accurately.
Interested? For more information about publication bias, check out Ben Goldacre’s talk at September’s TED (Technology, Entertainment, Design) conference:
Alternatively, have a read through the reviews used to write this article:
“Bad Pharma”- by Dr Ben Goldacre.
Written by: Andrew Balmer
Image credit: BMJ Group
This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.