Problems with evidence-based medicine aren’t a license to make stuff up

In the paper that brought the idea of evidence-based medicine (EBM) to prominence, Professor David Sackett et. al. wrote that

Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research. By individual clinical expertise we mean the proficiency and judgment that individual clinicians acquire through clinical experience and clinical practice. … By best available external clinical evidence we mean clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into the accuracy and precision of diagnostic tests (including the clinical examination), the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens.

In short, EBM is about integrating the best available research knowledge with the expertise of clinicians, who might – and often do – spot something that a model or a manual might not recognise as significant. As much of a naturalist as you might be – and I’m a very committed one – raw data exists in a context, and the context might often be a significant clue in telling you what data are relevant, and how they should be interpreted.

downloadSo I share some of Dr. Malcolm Kendrick’s concerns, when he writes that “EBM is now almost completely broken as a tool to help treat patients” thanks to the “evidence” being susceptible to corruption by vested interests, and pharmaceutical companies in particular. If you fund enough research, and suppress results that you don’t like, it’s certainly possible to end up with all the “evidence” pointing in a favourable direction. Favourable for you, that is, but not for the patient.

But the fact that something is funded by a pharmaceutical company doesn’t guarantee bias. There’s a difference between being cognisant of potential biases, and writing something off in advance, just because of whence it came. That sort of pre-emptive dismissal is a logical error called “the genetic fallacy“, and you can tell when it’s being committed if someone stops paying attention to the evidence at all, or starts claiming that they don’t need to even bother doing so. Like this, perhaps (from Kendrick’s post):

Some years ago I stated that I no longer believe in many research papers that I read. All I tend to do is look at the authors, look at the conflicts of interest, look at the companies who sponsored the study, and I know exactly what the research is going to say – before I have even read the paper.

I have also virtually given up on references. What is the point, when you can find a reference to support any point of view that you want to promote? Frankly, I do not know where the truth resides any more. I wish to use evidence, and the results of clinical studies, but I always fear that I am standing on quicksand when I do so.

We are at a crisis point. Medical research today (in areas where there is money to be made) is almost beyond redemption. If I had my way I would close down pubmed, burn all the journals, and start again, building up a solid database of facts that we can actually rely on – free from commercial bias. But this is never, ever, going to happen.

It’s rather alarming to see the person responsible for writing The Great Cholesterol Con – and for encouraging most of us to stop taking statins to lower cholesterol – professing “I do not know where the truth resides any more”. (If he really means that, a career in anthropology rather than medicine might suit him better, I’d suggest, in that he’s already learnt a key mantra of the field.)

Another peculiar thing you’ll find on his website represents quite a cunning stratagem. You see, he’s talked himself into a bit of a bind with this “don’t trust The Man” stuff – if he ever wants to sell you something like a drug, how could he offer you “evidence” in support of it’s efficacy, assuming that he or someone else with a vested interest was involved in that research?

Easy – by redefining what conflict of interest means. For him only, mind you – not for others, where it means that you can’t trust them, and don’t even have to read them to know that you can’t trust them. In his “Disclosure of Interest” page, he notes “I have become the medical director for a company making a heart health supplement called ProKardia”, for whom he does paid consultancy work. And here’s the cunning bit:

If I do write about ProKardia or any of the ingredients in ProKardia, in a positive light, you need to know that I have a financial interest. I did not use the word conflict of interest in this statement, as I do not believe I have a conflict. I have become involved in developing, and using, a product that I entirely believe to be a good thing.

Got that? He has a financial interest alone, but no conflict. Authors of papers with connections to something he doesn’t trust, or research sponsored by pharmaceutical companies, always represent a conflict – and you don’t even have to read the paper in question to know this. (Which, amusingly, means that even if they were to try to insert the same Humpty Dumpty clause into their papers, Kendrick would never read that either.)

I say “Humpty Dumpty clause”, because the book Alice in Wonderland contains this fabulous line: “‘When I use a word,’ Humpty Dumpty said in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.'”

It’s the same causal relationship to what words – and evidence – means that allows for claims that sugar is addictive (as addictive as cocaine, according to some), or that Harvard have endorsed low-carb high-fat diets. They haven’t – they’ve just said that carbs can sometimes be worse than fats, particularly the “bad”, saturated fats – yet the LCHF proponents cherry-pick this as support, even though the same group are explicit that the type of fat matters, and that animal fats remain something to be cautious about.

Or, it’s this casual relationship that allows for claims that Sweden has officially adopted the LCHF diet, even though the report – now finally available in English (the summary, at least), though that didn’t stop English speakers like Tim Noakes from relying on blog sources as authoritative on what it said – actually says (my emphasis):

in the short term (six months), advice on strict or moderate low carbohydrate diets is a more effective means of achieving weight loss than advice on low fat diets. In the long term, there are no differences in the effect on weight loss between advice on strict and moderate low carbohydrate diets, low fat diets, high protein diets.

I realise that this is just one element of the debate, which includes diabetes, heart disease and so forth. But these are two examples of scientists cherry-picking data that happens to support something they want to argue for – or that can be made to appear to support it, so long as you whip up enough hysteria, or persuade them that there’s some conspiracy afoot.

As Mark Wallace points out in piece for The Guardian on “addictive” sugar, “if the issue is people failing to act responsibly, then it won’t be rectified by treating them like children”. To that, we can add “treating them like fools”.

And when the same people doing the cherry-picking are giving themselves licence to simply ignore research they don’t like, and to dismiss conflict of interest issues (in their case, but not for you) with a regal wave, they stop being scientists, and start becoming shills.

By Jacques Rousseau

Jacques Rousseau teaches critical thinking and ethics at the University of Cape Town, South Africa, and is the founder and director of the Free Society Institute, a non-profit organisation promoting secular humanism and scientific reasoning.