Tooth fairy science
Think back to when you were a child, when anything and everything was possible. You wake up one morning with a deep ache along your jaw. You poke around with your tongue, and find there is a painful hole where your tooth used to be.
The pain, however, is dampened by the knowledge of the reward hidden under your pillow. This isn’t the first time you have lost a tooth, and your expert pattern recognition skills (along with the explanation provided by your parents) tell you that missing teeth, Tooth Fairies, and money under the pillow are always positively associated.
As a curious young investigator, you decide that you want to do some research on this Tooth Fairy. You might study the relationship between the type of tooth lost and amount of money the Tooth Fairy trades for it, or maybe whether there is a change in the amount of money the Tooth Fairy leaves behind depending on how many teeth you have already lost.
The possibilities for investigation are really endless, and might even result in data that is reproducible and statistically significant. This data, however, does not teach you anything meaningful because you failed to take a crucial step:
This illustration, which was presented by Harriet Hall, MD in Tooth Fairy Science and Other Pitfalls: Applying Rigorous Science to Messy Medicine, is used to highlight some of the concerns regarding the current practice of evidence-based medicine (EBM), the importance of scientific plausibility, and how science-based medicine (SBM) can get us back on track.
The problem with evidence-based medicine (as currently practiced)
Before the skeptical and science-based foundations of EBM, medical treatments were ruled by tradition, authority, intuition, and anecdotal experience – all of which are terrible approaches to treatment for (hopefully) obvious reasons.The paradigm shift towards EBM was an attempt to correct for these ineffective and often dangerous treatment approaches.
This was certainly a step in the right direction, as rigorous clinical trials are necessary for reducing bias. Levels of evidence were therefore established in order to distinguish research that focused on establishing first principles, and those establishing efficacy or effectiveness.
But as the random controlled trial (RCT) became the gold standard, there was also an (unintentional) relegation of basic science establishing plausibility. The problem arises from this weight placed on the clinical trials.
Clinical trials should always be, but are often not, based on a foundation of basic science establishing plausibility, and though plausibility is never sufficient for a treatment approach, it is a necessary first step.
There is a tendency in physical therapy to ignore or disregard scientific plausibility, while focusing almost entirely on trial results, and this is the central concern regarding EBM as currently practiced.
What happens when we ignore plausibility
By placing such an emphasis on the RCT and outcome evidence, many treatments studied and implemented by physical therapists bypass the important step of considering prior probability and scientific plausibility, and therefore drift towards very questionable pseudoscientific territory.
Again, there is no question regarding the importance of outcome evidence, but this evidence is not sufficient, and cannot be contrary to the entirety of scientific knowledge.
Dr. Jason Silvernail expands on the problems with focusing entirely on outcome only evidence in his article discussion (I highly recommend reading the entire discussion thread) on BodyinMind.org:
“We should focus as much on challenging our explanatory models and teaching in ways congruent with actual human physiology as we do on producing outcomes research, or the next generation of DPTs will be just one or two RCTs away from doing Reiki and Therapeutic Touch. Hey, if there’s "evidence to support it," it must be good, right? Can you see the problem with outcomes-only evidence?”
Scientific plausibility asks whether the model of treatment even makes sense based on what we know from foundational science, such as human physiology. If we started from this point of plausibility when evaluating, for example, the use of therapeutic ultrasound for pain, we would almost immediately conclude that based on what we know about the neurophysiology of pain, ultrasound will have little to no effect on a patient’s pain.
By ignoring the importance of plausibility, we fall dangerously close to embracing alternative medicine in practice, or what Dr. Hall calls Tooth Fairy Science in research. It should be noted that plausibility is not binary; it is a scale. Many physical therapy treatments are certainly more plausible than Reiki or homeopathy, but some are not so far off.
In a separate Evidence in Motion blog post, Dr. Silvernail further explains how focusing solely on outcome evidence can lead us closer towards treatments bordering on alternative medicine:
"I think it’s important to consider that, for MFR and most alt-med treatments, the "outcome evidence-only" brand of EBP won’t get us very far – and in fact may set us up for trouble.
Let’s say for example I publish an RCT showing that MFR produced clinically meaningful changes in an outcome measure of interest versus a competing intervention. Does that tell us ANYTHING about the truth of stored memories, fascial restrictions, or energy medicine? No. Success in the treatment DOES NOT validate the theory.
When I see my colleagues approaching alt-med treatments asking for outcome evidence, I get justifiably nervous – are they just one RCT away from believing in energy medicine? What we should be focusing on is the absolutely indefensible theory here – it’s scientific reasoning that will help us here, not statistics. Let’s never forget that."
Science-based medicine – an enhancement, not a replacement
As mentioned earlier, it was never the intention of EBM to focus on the clinical trial at the expense of scientific plausibility. It is generally assumed that there is a logical progression from first principles to applied medicine, but this is often not the case, and SBM is simply stating that we should place more emphasis on foundational science.
Perhaps one of the most straightforward ways to embrace the spirit of SBM is to raise our bar for biological plausibility. A deeper understanding of human anatomy and physiology can render entire treatment concepts indefensible, and help us focus on what works, and what should really be explored further through research.
Science-based medicine asks that we restore a skeptical perspective, and use our entire body of scientific knowledge when viewing treatments or claims, instead of asking if there are RCTs to support them. SBM is not a replacement for EBM, and acknowledges that prior probability and scientific plausibility is never sufficient for justifying a treatment, but it is a necessary first step. Justifying a treatment based solely on plausibility is a pseudoscientific practice.
Paul Ingraham best summarizes the goal of science-based medicine with the quote
“The idea of science based medicine is not to compete with evidence based medicine, but a call to enhance it with a broader view: to answer the question “what works?” we must give more importance to our cumulative scientific knowledge from all relevant disciplines.”
Key Points
- Before evidence-based medicine, most treatments were ruled by tradition, authority, intuition, and anecdotal experience.
- The paradigm shift towards evidence-based medicine was an attempt to correct for these ineffective and often dangerous treatment approaches, and has been, on the whole, a great step in the right direction.
- However, evidence-based medicine (as currently practiced) has placed so much weight on clinical trials, that basic science establishing plausibility has been all but forgotten.
- By failing to start at the question of plausibility, we fall into the trap of studying what Dr. Harriet Hall calls Tooth Fairy Science: studying effects of something that isn’t actually there.
- In practice, ignoring plausibility leads us dangerously close to embracing the use of borderline alternative medicine treatments.
- Science-based medicine can be viewed as a re-framing of evidence-based medicine, and asks us to consider the scientific plausibility and prior probability before embracing a treatment.
Resources to Learn More about Science-Based Medicine
To learn more about skeptical medicine, I highly recommend these following resources:
