When I was in college, the most shocking aspect of science was the degree of uncertainty that existed even at the highest levels of biomedical research. In high school, science was presented as black and white: plants photosynthesize, the heart has four chambers, etc. Mechanisms were either known or unknown. I never expected that turmoil brewed just beneath the surface of landmark discoveries. How could so many smart individuals each do so much work and come to different conclusions, only for some of them to be entirely incorrect? Take for example the anticancer small molecule indisulam. In 2004, an Italian group had done extensive work to demonstrate why they believed this molecule targets the enzyme carbonic anhydrase (Abbate et al) but a later Science paper in 2017 (Han et al) proved that this molecule in fact kills cancer cells by recruiting a splicing factor for degradation in cells. It wasn’t as if the previous Italian group had simply had a scatterbrained thought, their experiments truly suggested that indisulam might target carbonic anhydrase. There was no evidence of falsified data, the first group simply had not come to the correct conclusion.
Fast forward to March 2020. Shelter-in-place was turning into endless phone doomscrolling on my phone and relentlessly consuming all the COVID-19-related content I could find. I read comprehensively about everything from mechanisms of viral transmission (remember when the respiratory droplet idea was prominent?) to the latest biochemical papers detailing the structure of the spike protein on the surface of COVID-19 virions.
Despite all my knowledge, I gained a glimpse into the difficulties of public health messaging through conversations with close family members. They were getting some of the same basic information I was getting and, unfortunately, more. As the family member closest to the medical field, I fielded questions ranging from, “What type of mask is best?” to “Do we need to wipe down groceries with disinfectant wipes?” I would do my best to research the answers to these questions but would frequently feel dissatisfied with the answers I was able to give. I was rarely able to speak in absolutes after exhaustive reading. In response to “How far away from people should I be at the grocery store?” I was left to deal with a WHO report that did not cite aerosols as a major transmission route but then had to hold that report in one hand while other newer studies were reporting that COVID-19 virions could survive in the air for more than three hours (van Doremalen et al 2020).
While discussing the conflicting stories about COVID-19 prevention and transmission, a graduate school colleague joked that the general public was finding out just how chaotic the scientific process could be. News reports repeatedly highlighted growing public mistrust of new scientific reports about COVID-19 in part due to how the narrative and data surrounding the disease frequently changed. As frustrating as the actions of active and even moderately active anti-science/anti-vaccine campaigners can be, in truth I find it hard to fault non-scientists for seeing inconsistent information from individuals deemed “experts” and concluding that these experts are untrustworthy. I had trouble parsing all of the conflicting recommendations as someone with some medical training, so it was no surprise to hear of so much confusion amongst the general public.
In science, we try to avoid such trial-and-error conclusions by designing the most rigorous experiments possible. However, these standards are near-impossible to replicate in actual real world studies. It is very difficult, almost impossible, to control variables in studies of viral transmission to the degree necessary. And sometimes mistaken conclusions are inevitable despite our best efforts to avoid them. This is what why co-submissions in science can help get papers accepted. Co-submission is when two groups independent arrive at the same conclusion then submit their respective papers together to a journal citing the concordant independent findings as a major strength of both groups’ work. One group might be able to fool itself but two? Less likely.
Perhaps this “strength in numbers” tactic can help physicians and other medical professionals to parse through conflicting data. Sometimes we are pitting two studies that say one thing versus two studies that say the contrary, but more often than not, there are noticeable trends regarding conclusions that turn out to be accurate: They 1) have a way of sticking around until people start paying attention and 2) there is likely something interesting present if multiple groups have done studies with COVID-19 virions. And when physicians are done parsing, in my limited experience I have found that presentation can really make a difference in how information is received. In my personal experience I have found it helpful to go one step further when talking to my family and give short explanations as to how studies were conducted and how conclusions were reached. Brief, non-jargon-filled explanations can do a great deal in order to convince others to come around to your point of view because in the end, we are all just humans doing our best to deal with the information overload in a stressful, hectic time. Sometimes it can help to emphasize that we all, including scientists and physicians, are simply doing the best we can.
How do you explain unclear or conflicting study results to those not in a scientific field? Share your tactics in the comments.
Vicky is a MD/PhD student at UT Southwestern Medical Center currently in the process of obtaining her PhD in Biological Chemistry. Her areas of interest in writing include health policy, social determinants of health, and the intersection between basic science and the world of clinical medicine. When she's not in the lab, she enjoys reading, hiking, cooking, yoga, and hanging out with her dog, Dobby. Vicky is a 2021–2022 Doximity Op-Med Fellow.
Illustration by April Brust
