People often ask me who my favourite Doctor Who is. This isn’t as left-field a question as it may initially appear, as I’m pretty open about my status as a long time Doctor Who fan (the answer, in case you’re interested, is that I don’t have one; I adore them all).
The Doctor is one of those characters sometimes associated with the cultural trope the ‘insufferable genius’; a fictional character who is often difficult, selfish, and obnoxious but who is tolerated because they are great at what they do. The website TV Tropes carries a far more extensive range of examples than I could muster alone, and it was while browsing their website recently that I spotted Adam Conover on their ‘insufferable genius’ list. My defensive rant about why the Doctor does not belong on this list, and indeed why so many characters of this stripe are coded autistic, will have to wait for another day.
The version of Adam Conover they refer to is the lead character from the TruTV series Adam Ruins Everything, as played by his real-life namesake, the American comedian Adam Conover. In Adam Ruins Everything, the fictional Conover makes it his mission to disabuse people of common misconceptions, clubbing them over the head energetically with the truth — to their initial disappointment but eventual gratitude.
The last time I intersected with Conover’s work was a few years ago, when the Adam Ruins Everything companion podcast published an interview with Dr Kathryn Hall from Harvard Medical School’s Program in Placebo Studies. Any initial hopes I might have harboured about Adam Ruining the Placebo Effect were short lived, however, as the coverage of the topic was unfortunately wide of the mark.
During the interview, Dr Hall tells Conover:
There’s a growing problem in biopharma and clinical trials where it’s getting more and more hard to beat the placebo response. And this is costing the investors in pharma billions of dollars, [and] it’s leaving patients without treatments
Frankly, I found this to be an astonishing statement from a molecular biologist. If a novel intervention cannot improve on the placebo, the thing you’re “leaving patients without” is a treatment which didn’t work anyway.
Clinical trials are designed to test the specific effect of some intervention. The reason a control group exists is to try and capture as many uncontrolled and non-specific effects as possible — biases, regressions, learned responses, etc — so we can subtract them from them the effects measured in the active treatment group, and thus confidently attribute what is left to the specific effect of that intervention.
That it is “getting harder to beat the placebo,” says only that we’re getting better at capturing those non-specific effects within our control group. One possible reason for this might be better trial design, or better enforcement of existing protocols. This is actually a good thing for patients. Without rigorous controls, we would see drugs coming through clinical trials because of false positives, where patients in the active treatment group appear to improve, but not because of the intervention. Rather the improvement is because of something else going on within the trial; something which may not be there when the same intervention is later used in a clinical setting.
Dr Hall then goes on to talk about placebo effects in surgical interventions:
There have been two notable studies recently, where the surgical procedure failed to beat the placebo sham surgery. […] In the case of the most recent study on shoulder surgeries, a lot of people have these bone spurs that can develop in the shoulder, and they will do arthroscopic interventions where they go in and they use a scope to see where they can remove these growths and clean up the area. And in some cases they do some modest repairs. And in the sham surgery, they go in, they scope it, they just don’t do anything.
To which Conover asks:
When they do that, they found that, in some cases the placebo surgery is as strong or beats the real surgery? Is that what you’re saying?
And Hall responds:
It doesn’t beat [it], there was no statistically significant difference between the sham surgery and real surgery.
Conover is astonished:
Wow! That goes way beyond what I and I think most people think of the placebo effect as being capable of.
The paper I think Dr Hall is referring to is ‘Arthroscopic subacromial decompression for subacromial shoulder pain: a multicentre, pragmatic, parallel group, placebo-controlled, three-group, randomised surgical trial’, published by The Lancet in 2018.
Patients from thirty-two hospitals across the UK who reported shoulder pain for at least three months, were randomised into three groups. Patients all had previously completed a non-operative management programme, including exercise therapy, and steroids. Patients also were excluded if they had a torn rotator cuff.
Group A, consisting of 106 patients, were assigned to receive decompression surgery. This is where they insert a scope into the shoulder, and reshape the shoulder blade by removing material, including removing bone spurs.
Group B, consisting of 103 patients, were assigned to receive arthroscopy only. This is where the scope is inserted into the shoulder, but no other procedure is performed and no material is removed.
Group C were assigned no treatment; they were told they were on the waiting list for an operation, but they didn’t have any surgery.
The results showed there was no significant difference between the arthroscopy-only and decompression groups. That is to say, there was no difference between the sham surgery and the real surgery. Both were better than no treatment, but they weren’t better than each other.
While Dr Hall cites this as an example of the amazing power of the placebo effect, the study actually concludes:
The findings question the value of this operation for these indications, and this should be communicated to patients during the shared treatment decision-making process.
In other words, it shows this operation is a waste of time.
This is unfortunately common amongst advocates of a powerful placebo effect. What they hail as a win for the amazing power of the mind over the body, is in fact a study which demonstrates that this intervention is not effective, since it works no better than a sham control.
But, both sham and real surgery were better than no treatment, right? So doesn’t that show the fake surgery did improve something? Well, yes and no. From the paper:
Surgical groups had better outcomes for shoulder pain and function compared with no treatment but this difference was not clinically important.
Although the surgical groups were statistically better than no treatment, this does not translate to any meaningful clinical outcome. The patients were technically improved, but practically it made little difference. Moreover, the paper says:
The difference between the surgical groups and no treatment might be the result of, for instance, a placebo effect or postoperative physiotherapy. [emphasis mine]
Yes, it turns out that following surgery, the patients in both the sham and real surgical groups were given postoperative physiotherapy, and this might be responsible for the (minor) improvement in the surgical groups, compared to no treatment.
What Adam Conover exclaims is a “much stronger effect than I thought” actually turns out to be a clinically irrelevant, if statistically significant, difference from no treatment, which can be explained by a parallel intervention — that being post operative physiotherapy.
As is sadly the case with so much other research in this area, the claims made by advocates of the ‘powerful placebo’ fail to stand up to scrutiny.
