Am I a Fideist?
I don’t think so, but Scott Alexander suggested I was in a blog post he wrote in response to some of my critical tweets on how he had covered the ivermectin ‘controversy’. Scott mentions he has been looking into YouTube streamers and by coincidence so have I. He is right that internet beefs are a good way to drive engagement but since we are not on YouTube maybe we can avoid the pitfalls of that platform. I mean look, here I am writing a Medium article, many years after my last one.
In the comment section afterwards, Scott acknowledged he wrote the response while a little heated and I think this may account for some of the more extreme interpretations of what I said. But my original post was worded harshly, so I would be smashing up my glass house by complaining about tone. Live by sarcastic digs and you may occasionally receive them in return!
In any case, while some of our disagreements might come down to temperament and misreading of arguments, I also do think there are areas where we substantively disagree and that it would be useful to explore why. I will also acknowledge that there is likely no single correct answer to some of these issues. Ultimately, it will come down to individual cost/benefit analysis and values. So I’m not anticipating convincing Scott or the majority of his audience that I am right. But maybe I’ll get some to see some merit in my arguments and demonstrate that I am not actually a closet Fideist. Or at least I hope not.
Scott’s blog post was helpfully numbered, so I will try to follow suit in my response to keep the points I am addressing (relatively) clear.
1. On the search for Atlantis
Like Scott, I read Graham Hancock when young and found some of his arguments compelling. According to Scott’s account, he could not locate any good quality criticism with skeptic blogs just hosting “a thousand identical pieces saying “lol you’re stupid and racist if you believe this, haven’t you heard that conspiracies are always wrong?”. This resulted in Scott pulling himself up by his own bootstraps, learning about geology from the ground up, scuba diving in alleged ruins, and going on what ultimately amounted to “a five year wild goose chase”.
Scott reports this being a valuable learning experience and the scuba photo does look awesome! He also explains he is not resentful of Hancock. Despite coming to doubt his claims, he regards him as a “weird, well-meaning guy who likes pyramids a little too much”. But he does report “a burning anger” directed against “anti-conspiracy bloggers, anti-conspiracy podcasters, and everyone else who wrote “lol imagine how stupid you would have to be to believe in Atlantis” style articles”. As he notes, a kinder approach could have saved him from a bunch of effort.
My experience was very different. I did find Hancock’s claims compelling initially and was shocked at the seeming close-minded nature of the experts he described. But when I started looking into the critics, I found what Scott did not. I found a bunch of detailed criticisms from the skeptic community, relevant experts, and even BBC documentaries. Some of these (I think) I discovered from following disparaging comments on Hancock’s website, forums, and an addendum in his book. This was the early internet era and I might be a bit younger than Scott, so maybe when I looked the critics and expert responses were easier to find. But I did locate them and did not find them to be insulting and dismissive. As a result, I had a rather different experience than Scott.
Sadly, I did not end up scuba diving in ruins or studying geology in any great depth (what I knew about geology was mostly derived from debating with creationists). What I did learn about was how superficially compelling it is to present alternative histories as forbidden knowledge and that the actual history was drier but more interesting and complex. Like Scott, I did not come away resenting Hancock or the people that found him convincing. I was one of them after all. Indeed, I think Hancock is both sincere in his beliefs and good at making his arguments in a compelling way for a lay audience. I did, however, gain an appreciation of the frustration of the experts he disparages. Many of them have dedicated their careers to studying history that Hancock casually dismisses or misrepresents.
I also don’t think I would have the same intuitions as Scott that personally exploring the ruins would be informative. I feel like that would likely skew my perspective further, as a powerful emotional experience could deliver inaccurate intuitions. Moreover, for such an effort to be properly informative it seems like it would require me to already possess the expertise to properly assess what I was seeing. Scott was better at resisting confirmation bias than this and the experience ultimately helped to increase his doubt in the Atlantis thesis. It is also almost certain he had the more fun experience. So judged solely on experiential grounds, his approach was likely more rewarding!
It is interesting that we both took rather different lessons from our early encounters with Hancock. Nonetheless, a place where we both agree is that there is value in detailed rebuttals produced by experts or well-researched amateurs that are laid out in a way that a curious lay audience can understand. Encountering them helped me and I would be a hypocrite to disparage people who make such efforts. Indeed, the podcast I co-host does lengthy deep dives in efforts to critically evaluate the content and rhetoric of (potential) modern guru figures. So this would be an odd stance for me to take. This leads me to point two.
2. On Doing Your Own Research
Scott was a bit off in accusing me of devoting my career to ivermectin and related topics. Actually, this would have worked better as a subtweet of our lack of coverage of the topic, since my co-host and I recorded an extended interview with Gideon MK on the criticisms of the Together trial months ago and have still not edited it together yet! He is on firmer ground, however, when noting a preoccupation with the Weinsteins, which I think was the broader spirit of the zing.
Here, it is absolutely true that on the podcast and on my Twitter feed, I do often reference the Weinsteins, other secular gurus, and conspiracy theorists. It is not the main topic of my academic career, but certainly is the main topic of my minor public profile. But that is, I would contend, something of a feature, not a bug. I am interested in modern secular gurus and the Weinsteins are a super stimulus for the category. For all the gurus we have covered we have rated each of them on ten recurring guru characteristics, that we tongue-in-cheek label as a ‘Gurometer’. To date, no one has scored higher than Bret and Eric Weinstein. Only Reverend Moon, from the Moonies, has tied with them. So Bret certainly is referenced by me often … too often? Maybe! But in my defence, he is impossible to parody. I mean just look at this…
And when it comes to Ivermectin debates, Bret is also the elephant in the room. He was one of the first high-profile figures to promote it: describing it as a near-perfect treatment and a viable cheap alternative to vaccines. He also alleged that the evidence was being suppressed by evil pharmaceutical companies (and other shadowy cabals) and that this constituted ‘the crime of the century’. He introduced Pierre Kory and later Robert Malone and Peter McCullough to Joe Rogan. Alexandros Marinos emerged from the DarkHorse fandom, organising his ‘Better Skeptics’ project (for a critical evaluation see here) that ‘proved’ that Bret, Steve Kirsch, and Robert Malone made no substantial errors when discussing vaccines and Ivermectin.
All of this demonstrates that Scott is correct that I know far too much about Bret’s activities. But I think this is also important contextual information. It is worth considering whether, without things like Rogan’s emergency podcasts, widespread ‘heterodox’ hostility to anything deemed ‘establishment’, right-wing media appearances by contrarian doctors, and hyped-up conspiracies on podcasts, ivermectin would have been on the public (or the rationalist) radar? My answer would be no. And I think that matters for how skeptically you should approach the claims being made.
Is ivermectin a drug that could have some efficacy in treating covid? Possibly. But I did not see many experts saying otherwise. What I did see was experts saying the evidence was not there to justify the hyperbolic claims emanating from the ecosystems above, including the inaccurate perception that it was an effective alternative to vaccination. To me, this is the salient context that it would be wrong to ignore or downplay. One practical consequence of this is that a modification to credibility should be applied if, for instance, an individual’s interest in the ivermectin topic comes from their devotion to a conspiracy-prone podcaster and their extended network of guests.
More broadly, Scott interpreted me as saying that to “even try to ‘evaluate the evidence’ at all is a mistake… you should admit that some people are idiots who believe things there’s no evidence for, and move on” but that is not my argument. My position is rather that if you do decide to ‘do your own research’, evaluate the primary literature, and conduct your own meta-analysis, you have to do so while being honest about your capabilities (and the capabilities of others). Have you read clinical trials before? Do you understand statistical analyses? If not, or your abilities are limited, then you should adjust your confidence accordingly.
It is also true that, on average, I think most people online would be better served by not attempting their own meta-analysis of a field they are not familiar with, but rather by learning how to identify reliable expert sources and whether there is a general consensus in a field. Scott is a smart cookie and someone who knows how to interpret studies, yet he still made errors in his analysis due to this not being his particular field of expertise. How much credence should we lend to the critical evaluations of someone else who only recently discovered what p-values are?
I am someone who read Scott’s initial long post and thought he did a good job of collating the studies and demonstrating how the ‘30 positive studies’ claim becomes much less impressive when you apply a basic critical lens to the quality of the studies. Around 2/3rds were deemed so low quality to be uninformative to include. So we are actually talking, at the time, of around 11 relevant studies of varying quality and size.
Now you might object, as Scott did, that I am putting the horse before the cart, relying on his analysis and not showing my work but my counterargument would be that the position he arrived at reflected the general consensus of experts. Namely, that the positive research literature was generally low quality and that you would need better evidence to make the dramatic claims advocates were, especially when we had vaccines and other effective treatments. Rather than just being casually dismissive, many credentialed experts had produced in-depth threads discussing the limitations and problems with various studies, others had offered higher-level assessments, and some had discussed the research on podcasts. Scott’s blog piggybacks on a bunch of these, with acknowledgements, including the heroic work of Gideon MK.
So my contention here is that contra to widespread claims amongst folks, like Bret Weinstein and Alexandros Marinos, that the mainstream research world was ignoring mountains of high-quality evidence for ivermectin, in actual fact, many experts had performed careful critical assessments and expressed their opinions publicly, eventually in the research literature, but before that on podcasts, blogs, and Twitter. I know this because I read and listened to a bunch of them and I do not think they were that difficult to locate. That some journalists wrote dismissive articles or that experts and institutions issued casual disparaging comments is true but also somewhat inevitable when you have a lot of experts. And especially when said experts are routinely dealing with the claims coming out of the anti-vaccine and covid-sceptical media ecosystems.
I also want to be clear that I am not saying it is only low-intelligence, credulous fools who lack any relevant expertise that advocated for ivermectin. Pierre Kory, Robert Malone, and Peter McCullough are credentialed experts. There are also many well-documented examples of a medical consensus being later proven wrong. This is why I am an advocate for Open Science practices, and related methodological reforms, and why I share the critical perspective of people like Stuart Ritchie, Ben Goldacre, and Scott Alexander(!) when it comes to the need to look critically at modern scientific research. It is also why we know that meta-analyses are in need of critical evaluation and are heavily reliant on the quality of the studies that they include. Advocates for homeopathy, acupuncture, and almost every other alternative medicine can point to positive meta-analyses, typically conducted by advocates or sympathetic researchers (in some cases including Cochrane reviews) but higher quality studies and more rigorous meta-analyses paint a different story.
What I would add here is that the very real problems we see in mainstream research, including publication bias, inappropriate control comparisons, and overhyped claims, are tripled when we look into the output of pseudoscientific, anti-vaccine, and conspiratorial communities. And just like how we can see warning flags in research literature, when there are inflated claims based on low-powered studies and p-values that hover suspiciously close to significance boundaries, I would argue we can identify a different set of warning flags around the figures and communities that were cheerleading ivermectin. Things like personal victimization narratives that present themselves as heroes silenced by an evil establishment, positive references to existing networks of anti-vaccine advocates, reliance on emotion-evoking anecdotes, demonisation of mainstream doctors, exaggerating the quality of positive studies, and so on. If you want to see an example of these flags being identified in practice, I would suggest listening to the episode we did on Robert Malone and Peter McCullough’s appearance on Joe Rogan.
So in summary, no my position is not ‘don’t do research’ and that ‘looking at evidence makes you a bad person’. My position is that when looking at evidence be aware of your limitations, look critically at all research but consider relevant contextual factors, and weigh expert opinion and consensus views in accordance with that. Scientists with Nobel prizes can and often do promote pseudoscience, and well-established effects can prove non-replicable when the methodological standards of a field are too low. This is all true but it is also true that conspiracy theorists/communities and fringe researchers do exist and they will often devote huge efforts to defend and promote inaccurate claims.
3. On Trusting the Media
At the end of the article, Scott discusses a controversy surrounding whether ‘premenstrual dysphoric disorder’ was culture-bound or not and notes the following:
An article in Slate had claimed that it was. It cited five studies to this effect, and argued that the case against originated from the bad old sexist tendency for male doctors to diagnose women with “hysteria”.
But then an article in Vox claimed that it wasn’t! It also cited about five studies to this effect, and argued that the case in favor originated from the bad old sexist tendency for male doctors to dismiss women’s lived experience.
So if you Trust The Media, and Trust Science, and don’t want to fall for sexist myths, which side do you choose?
The correct answer is that by even considering this question, you’re in a state of sin. You choose the side with more scientific evidence and better studies, obviously.
My answer to the question Scott poses here is that you do not actually have to pick a side on an issue you have only learned about from two articles summarising some literature on a topic you are unfamiliar with. I do not outsource my understanding of science very often to Vox and Slate (though I think Brian Resnick’s articles are often good). So what I would do here is likely make a mental note that this is a topic that there is some debate around.
If I was sufficiently motivated I could then go and dive into the primary literature and try to assess the validity of the studies myself but I likely would look first for review articles and try to identify what factors are impacting the assessments. If one set of scholars advocating an interpretation holds a particular theoretical or political set of values and the opposing group a different set that would also be informative. It would also be very relevant if one side was being promoted heavily by figures from conspiracy theory ecosystems and/or hyperbolic media coverage. It would not be a ‘sin’ to dive into the literature and try to identify which side has the stronger evidence in support. But, as above, it is something that should be done with due acknowledgement to your familiarity with the topic, the research methodologies involved, and your own abilities.
I will finish my own indulgent response by saying I co-sign Scott’s concluding call for tolerance and the need for science communicators to engage with controversial topics and provide explanations for their stances. I would just add that I also understand why many would be frustrated or reluctant to do so when there are many highly motivated individuals and well-supported ecosystems that will relentlessly attack them for doing so and whose assessments of an issue are predetermined.
