Is Moncrieff’s Chemical Imbalance Paper a Decent Systematic Umbrella Review? Part 1

Learn how to assess the validity of an umbrella review before we delve deeper into Moncrieff’s claim that serotonin theory is dead and so depressed persons should avoid meds.

“Chemical change, you over shouted the warning…” - Grace Slick

It’s easy to get lost in the flood of publicity that Joanna Moncrieff and her colleagues whipped up for her chemical imbalance systematic umbrella review. With obvious pride, Moncrieff reports her review smashed records for the immediate attention it received, like being accessed at the Nature Molecular Psychiatry website a phenomenal 413,000 times.

Scientific breakthroughs do not get that kind of attention without the manipulation of social media.

This publicity campaign seriously distorts the search results anyone obtains entering <serotonin> or < chemical imbalance> in an engine like Google Chrome. Articles and blogs and even tweets from whoever got swept up in Moncrieff’s anti-psychiatry/critical psychiatry parade dominate the results of a search, at least for a while.

Moncrieff’s SUR team had to have started well before December 2020, the time-stamped registration date of her protocol for the review. Her group had lots of time to prepare nonacademic articles that would actually be more important for her surprise attack than the SUR in the prestigious Molecular Psychiatry itself.

It is a mindless, unthoughtful frenzy on social media. Few of Moncrieff’s even most hardcore followers tweeting in the middle of the night have read the mind-numbing systematic umbrella review (SUR) on their smartphones, rather than just the tweets or the much lighter-weight articles about the SUR on The Conversation or The Guardian.

Let’s get serious. Who really clicks on the links in Tweets and accesses and reads scientific articles before liking and retweeting tweets, despite Twitter’s lame warning that we should do so?

The startled psychiatrists Moncrieff sought to embarrass can get off some kneejerk responses or maybe have some article lying around that was relevant that they could post.

Otherwise, it takes time for any thoughtful response to be assembled. Patients, clinicians, and policymakers have to wait.

To my knowledge, no one had yet addressed whether Moncrieff‘s SUR is even a decent effort. That is a vital question to answer before getting involved in what looks like serious discussion on social media but hateful trolls and bots are having a lot of undue influence.  

I decided to put together materials useful in assessing the aim and validity of a SUR. I will describe the steps involved so that my effort could be duplicated by anyone else.  Inevitably we will face another t such propaganda campaign disguised as a detached consideration of the biomedical literature.

An example of a systematic umbrella review

 I already had in my files a SUR article in BMJ from 2018.

O’Sullivan JW, Muntinga T, Grigg S, Ioannidis JP. Prevalence and outcomes of incidental imaging findings: an umbrella review. BMJ. 2018 Jun 18;361.

I recommend the article as a superbly done and well-described SUR, even if you have zero interest in incidental imaging findings.

Incidental imaging findings occur when clinicians identify something suspicious in an imaging test of a patient for whom they would not have other grounds to expect an abnormality.

Incidental imaging findings serendipitously diagnosed in an asymptomatic patient or symptomatic patient undergoing imaging for an unrelated reason are fast becoming a modern medical crisis. The rapid rise in demand for imaging coupled with rapidly advancing image resolution is driving a surge in incidentalomas

Wikipedia is good enough for a basic definition of incidentalomas.

In medical or research imaging, an incidental finding (commonly known as an "incidentaloma") is an unanticipated finding which is not related to the original diagnostic inquiry. As with other types of incidental findings, they may represent a diagnostic, ethical, and philosophical dilemma because the significance is unclear.

The BMJ article explained why a SUR was needed right then.

There is large variability across different imaging techniques both in the prevalence of incidentalomas and in the prevalence of malignancy for specific organs. This umbrella review will aid clinicians and patients weigh up the pros and cons of requesting imaging scans and will help with management decisions after an incidentaloma diagnosis. Our results can underpin the creation of guidelines to assist these decisions.

If you have the inclination, read the sometimes technical article and get a sense of just how rigorous the authors are. They describe in excruciating detail the limitations of what clinicians think they know about incidentaloma, how the existing meta-analyses are contradictory and flawed, and how uncertain the authors are about their own conclusions. This is hugely valuable medical information that took a SUR to reveal.

General Resources for Systematic Umbrella Reviews

I stumbled upon a great general resource in a simple search on Google Chrome.

Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. JBI Evidence Implementation. 2015 Sep 1;13(3):132-40.

The open-access paper was very clear and from an institution known as an excellent source for information concerning the various kinds of reviews that one can find in the literature.

The abstract starts with

With the increase in the number of systematic reviews available, a logical next step to provide decision makers in healthcare with the evidence they require has been the conduct of reviews of existing systematic reviews. Syntheses of existing systematic reviews are referred to by many different names, one of which is an umbrella review. An umbrella review allows the findings of reviews relevant to a review question to be compared and contrasted. An umbrella review’s most characteristic feature is that this type of evidence synthesis only considers for inclusion the highest level of evidence, namely other systematic reviews and meta-analyses.

And ends with:

Umbrella reviews provide a ready means for decision makers in healthcare to gain a clear understanding of a broad topic area. The umbrella review methodology described here is the first to consider reviews that report other than quantitative evidence derived from randomized controlled trials. The methodology includes an easy to use and informative summary of evidence table to readily provide decision makers with the available, highest level of evidence relevant to the question posed.

The first citation in the JBI Evidence Implementation paper was:

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLOS Med 2010; 7: e1000326

Anyone who is interested in learning more can click on the link and see the authors’ concerns about the pile-up of too many redundant meta-analyses of inadequately controlled clinical trials. Rigorous SURs are the needed solution for many areas of medicine, not more inconclusive and overlapping meta-analyses.

With some idea of what a SUR was, I wanted to know how someone decides when a SUR was decent and useful as a basis for clinical and public health policy.

Undertaking a SUR is warranted by the quantity and quality of available evidence and some urgency to make sense of confusing conflicts among the findings.

I went back to the first page of the results of my Google search and found.

Fusar-Poli P, Radua J. Ten simple rules for conducting umbrella reviews. Evidence-based mental health. 2018 Aug 1;21(3):95-100.

The 10 recommendations are

“Ensured that the umbrella review is really needed, prespecify the protocol, clearly define the variables of interest, estimate a common effect size, report the heterogeneity and potential biases, perform a stratification of the evidence, conduct sensitivity analyses, report transparent results, use appropriate software and acknowledge the limitations.”

Conducting a SUR is a process that must begin with a clearly specified research question based on how competent scientists are already defining it. Then someone committed to science undertakes some highly technical procedures to calculate some precise findings with an estimate of just how much confidence consumers should have in this precision. The authors of the 10 rules draw on their own work concerning risk and protective factors for psychotic disorders for examples. Joanna Moncrieff and her closest followers characteristically deny that psychosis even exists.

The 10 rules paper cites one of the authors of the incidental imaging SUR.

Ioannidis JP. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ. 2009 Oct 13;181(8):488-93

Again, the emphasis is on rigor and the threats to achieving it. Here is a table with some specific considerations.

Obviously, a SUR takes a team and a commitment to doing precise but tedious work.

Check out the CVs of Joanna Moncrieff and her co-authors, Ruth E. Cooper, Tom Stockmann, Simone Amendola, Michael P. Hengartner, and Mark A. Horowitz.

What capacity does the Critical Psychiatry Network have for conducting a systematic umbrella review? What track record do they have of such rigorous evidence-based thinking? In applying for a grant, what relevant previous work would be cited? I have been on lots of grant review panels in the US and Europe. I have never seen a proposal for a review anything like Moncrieff produced.

Check out the portrayals in popular media by Moncrieff of the results of the Molecular Psychiatry article. Then, how her legions of followers portray the SUR on Twitter.

On Twitter, they consistently portray psychiatrists as evil cryptoNazi miscreants who believe in eugenics and who deceive patients and poison them with drugs that are not only ineffective but dangerous. Psychiatrists are further ridiculed if they protest this portrayal is unfair or if they insist that they do not and never have subscribed to the notion that treatment of depression involves eliminating a chemical imbalance.

There is no sense of the nuance and humility required by scientific rigor in antipsychiatry groups’ Twitter attacks. Only wild-eyed social media radicals can be so certain in a world of uncertain scientific findings.

Maybe we can be done before we begin with our probing of the serotonin hypothesis/chemical imbalance SUR as a piece of serious scientific work.

We can conclude that there is little serious commitment to science among the criticals, but a wholesale rejection of what other scientists define as the state of their science. There is no history of meaningful engagement with those with whom they disagree.

No, let’s get on with the bloody task because a false sense of unanimous consensus has been created by the critical psychiatry propaganda machine of what the scientific findings are. As Monty Python would say, “And now for something completely different”—a nuanced, evidence-based discussion, even if it leaves a lot of people out.

Final thought—

I doubt that there are any other SURs in the literature that start with the assumption that the scientists who create a literature and the clinicians who follow it are so stupid and reluctant to give up on such a dead theory and the ineffective treatments that theory has inspired.

Are psychiatrists so like astrologists? I am sure that a considerable number of loyal followers of Moncrieff think so and that is why there is so much trouble trying to talk to her group.