Independent Peer Review of the Adverse Childhood Experiences Checklist (ACEs)
Why "Risk for cancer increases with the number of items checked" can't be right.
I needed an independent peer review of a published article extolling the ACEs as a means of predicting the modifiable risk of getting cancer in later life.
I found the email of a noted cancer epidemiologist on the Internet. Of course, I did not tell him about this egregious claim. If he was truly a noted cancer epidemiologist, he should be able to figure out many other things that were wrong with using the ACEs to predict mental and physical health.
I asked him to provide an independent peer review of an already published paper, Miller et al. “Adult health burden and costs in California during 2013 associated with prior adverse childhood experiences.”
I contacted him again a few weeks ago. We were amused to discover that neither of us could recall when and why I had originally requested the review.
He was the first to say:
“This is not something I've worked on. Are you confusing me with someone else?”
And then
“Now I remember -- I had no memory of my response to you from 2013. “
The mention of 2013 in the title also threw me off. Fortunately, the cancer epidemiologist recalled the actual date of publication (2020) that linked to the open-access paper.
Miller TR, Waehrer GM, Oh DL, Purewal Boparai S, Ohlsson Walker S, Silvério Marques S, Burke Harris N. Adult health burden and costs in California during 2013 associated with prior adverse childhood experiences. PLOS One. 2020 Jan 28;15(1):e0228019.
The cancer epidemiologist's critique applies to almost all articles reporting analyses of ACE data in large samples.
Why don’t we see many critiques, even letters to the editor, if the problems with the ACEs are so pervasive and easy to spot?
I believe many experts are afraid to criticize the questionnaire because of the way Kinderman, Lucy Johnstone, Eiko Fried, and David Healy—the Gang of Four savagely mobbed me and attempted to drive me from social media in response to my critique.
According to the gang, my heavily accessed blog was dangerous because it was just a fig leaf for my real agenda, promoting adult sexual exploitation of children, AKA pedophilia.
Please judge for yourself.
An independent peer review of Miller et al., Adult health burden and costs in California during 2013 associated with prior adverse childhood experiences
I (JCC) provide the cancer epidemiologist’s review with my italicized plain English commentary. He begins by noting that he did not go back to the granddaddy of all ACE studies, which was conducted with patients in the Kaiser-Permanente managed care system in California. He is assuming that we both know what is in that study, in terms of its findings and the original authors made of their findings.
What I have are mainly questions and some basic comments about the limitations of this “study.” I haven’t gone back to the study by Waehrer or the CDC-Kaiser-Permanente study, which this paper draws on. I’m assuming that you are familiar with the literature in this area.
1. This is a cross-sectional study examining the correlation between ACEs, on the one hand, with health behaviors and health outcomes. All data are self-reported. It is possible that reporting of ACEs is influenced by current circumstances. What one would want to see is a validated instrument used to assess ACEs in early life, rather than from the standpoint of adulthood.
My plain English interpretation: The ACEs is a self-report checklist used to predict self-reported current mental and physical health. We don’t know how much the adults are responding to their current situation, not the past. We need evidence that the adult ACEs is related to what happened in early life.
2. This paper is concerned with coming up for dollar amounts for the cost of ACEs in California in terms of health care costs and Disability-Adjusted-Life-Years in adulthood. In considering the questions and limitations pointed out below, one had to realize that any upward error in the estimation of the prevalence of ACEs and of the association of ACEs with disease outcomes will result in an over-estimation of ACE-associated health costs. When examining a poorly-defined phenomenon (ACEs) [see below] in relation to the cost of common diseases, one is going to come up with large, and attention-getting, dollar amounts.
The reviewer thinks the paper is intended to impress readers with its assignment of a dollar amount for not doing something about ACEs in California. ‘The paper is highly unusual in calling for routine screening of all patients an other members of the community because (os the arguemnt goes) NOT to screen everybody will miss changes to save lives. Furthermore, the authors are claiming to provide estimates is dollar amounts of how it is a good use of money to screen and save lives. The estimate is likely to be huge and attention-getting but wildly inaccurate. Hint: We may be venturing into politics and the authors’ self-promotion, far from what best science can conclude. Stay tuned.
3. Adverse childhood experience (ACE) is not defined. This is perhaps the gravest problem with the paper. Sixty-one percent of adults in California reported one or more ACE. This indicates to me that ACE appears to be a very amorphous category, including a wide range of events judged to be adverse. But there is no indication from the authors that events have to be rated in terms of severity. If everything qualifies as trauma and “toxic stress,” the terms become meaningless. (I had a father, who was an accomplished professor at a major university. When my brothers and I misbehaved, on rare occasions, he would take off his belt and strap us on the backside. He also shouted when he got angry. Today this would qualify as child abuse. Based on that experience, should I be considered to have an ACE? In fact, I had an incredibly fortunate childhood and was anything but traumatized by my father). The point is that any phenomenon present in 61% percent of the population would appear to be amorphous and ill-defined to the point of being meaningless. The authors claim to show that there is a “graded relationship” between number of ACEs and health risk factors/health outcomes, but I can’t judge how convincing that graded relationship is.
We need more to go on than say that adverse childhood experiences are whatever the checklist items measure. Most Californians endorse at least one item. Should we therefore conclude that something so common can be so traumatic, or is the difference meaningless between traumatic and non-traumatic experiences? The reviewer refers to the personal experience of having a father who got angry and spanked him. Yet, he had an “incredibly fortunate experience.” Whoa, should we get the cancer epidemiologist into a reviewer protection program? The gang of four will accuse him of the review being a figleaf for promoting paternal physical abuse of sons.
4. There is no mention in the paper (at least that I found) of SES, poverty, or educational level as potential confounders (or effect modifiers) of any relationship between ACEs and adult health behaviors and disease outcomes. For example, smoking shows a strong inverse association with education – in the U.S. smoking has become a behavior largely confined to those without a college education. SES also is likely to mediate many of the other associations examined in the paper.
The paper seems to assume that whatever the checklist measures is more important than class, poverty, or how much education someone’s parents had. There is evidence to suggest that these factors may explain negative health outcomes that the authors attribute to the ACEs checklist scores or that they may modify any effects the authors claim for the ACEs.
5. As far as I could tell, the analyses don’t appear to be multiply-adjusted for the different risk factors. So, confounding would appear to be a serious problem. In addition, when examining the association of ACEs with risk of cardiovascular disease, it would be important to take other CVD risk factors into account. CVD is the classic multifactorial disease. Risk factors include: male gender, age, high blood pressure, high serum cholesterol, current smoking, and obesity. To sort out whether ACEs make a contribution to CVD risk – and what the magnitude of that contribution is – over and above the presence of these major risk factors would require a much more detailed analysis using a large prospective cohort.
The reviewer looked at the paper's complex analyses and did not see any effort byt the authors to control for the known risk factors for cardiovascular disease. This is very basic clinical epidemiology 101.
6. As mentioned earlier, once one had finer-grained information of the prevalence of ACEs, in addition to correcting for confounding factors, one would want to look a range of modifying factors that could influence the association of ACEs with health outcomes (age, SES, family structure [intact family, siblings, etc.], religion, ethnicity, etc).
The authors did not pay any attention to the clinical epidemiology of chronic illness. They also did not take into account the family and social determinants of these risk factors. I would add ACEs research is not a great way to study the social determinants of health inequalities. It is a great way to pretend to study the social determinants of health and win the praise of others who are also confused.
So, I can’t avoid the impression that the authors’ goal in this paper was to apply the very vague and amorphous phenomenon of ACEs to common adulthood diseases and come up with eye-catching dollars amounts. This may draw more attention to the problem of childhood trauma but, unfortunately, by failing to make any crucial distinctions or qualifications, it is likely to promote well-motivated concern but will nothing to clarify the true magnitude or nature of the problem.
The reviewer believes that his statistically and methodologically sophisticated review allows him to speculate on how the authors might have an agenda that is less pure than promoting the best science of social determinants of health. Fair enough. He has earned that privilege by his wielding his expertise in clinical epidemiology, which is probably greater than for most people tempted to enter into this conversation, even the Gang of Four.
I went a bit wild, and I’m sure to have missed things, but my hope is that some of my questions comments will connect up with your insight into research in this area. Will be interested to see what you write.
I thought I did not write anything. This all was during COVID; my wife was among the first cases in Philly after we returned from Europe. She phoned the advice nurse at Penn, who told her to do nothing except monitor her temperature for 3-5 days. She said she had to decide whether to get back on the plane and fly to Arizona to tend to her chronically ill parents. The advice nurse told her it would be fine if she did not have an elevated temperature for five days. She flew to Arizona and landed with florid case of COVID-19. She isolated herself from her parents, and then she cared for them. She was in Arizona for months and eventually had to buy a car to drive back.
In all the chaos of lockdown and my village boarding up, I thought I did not write anything. Then I discovered an article that I had written for Medika Life. Stay tuned for a surprise.
I would mention that there is a literature on the influence of early life events on adult stature and the risk of disease in adulthood, but that literature strikes me as much more sophisticated and much more aware of the kinds of problems that need to be addressed in exploring a rich and complex area like this.
That is sophisticated, nuanced literature. It won’t attract as much attention as this funky PLOS One article and the publicity machine that brought it nationwide attention. I checked the altmetrics, which indicate the article had many views, but there were few citations four years later.
Postscript
Please subscribe to my Substack article to be alerted to the next installment. Make it a paid subscription if you find this narrative particularly interesting or realize that my writing causes me to get harassed, canceled, or worse. This disclosure suggests that I might need the money.