In the realm of medical, neuroscience, psychology, and educational research, the efficacy of various treatments, including drugs, cognitive training, biofeedback, and neurostimulation, has been under rigorous scrutiny. However, the diverse findings in these fields have led to growing skepticism among researchers (Lampit et al., 2014; López-Alonso et al., 2014; Sitaram et al., 2017).
Amid this landscape, neuromodulation has emerged as a promising treatment method, with the global neuromodulation device industry projected to reach $13.3 billion in 2022 (Colangelo, 2020). Notably, transcranial magnetic stimulation (TMS), a form of neuromodulation, has gained approval for treating conditions like migraine, major depression, obsessive-compulsive disorder, and smoking addiction in multiple countries, including the US (Grover et al., 2021; Khedr et al., 2005; McGough et al., 2019).
Despite the optimism surrounding neuromodulation, contradictory findings have surfaced, raising questions about its universal efficacy (Horvath et al., 2015; Medina & Cason, 2017; Parkin et al., 2015; Wang et al., 2018; Westwood et al., 2017). Various factors have been considered to explain this heterogeneity (Filmer et al., 2020; Guerra et al., 2020; van Bueren et al., 2021), yet a critical element has been overlooked—the role of subjective beliefs in treatment outcomes.
Objective vs. Subjective Treatment
Participants in TMS and tES studies often report perceptual sensations, potentially influencing their awareness of receiving active treatment. To address this, sham (placebo) protocols are employed, mimicking the sensory aspects of active treatment without affecting cortical excitability substantially. These sham treatments control for participants’ beliefs about the type of stimulation received (Fritsch et al., 2010; Nitsche & Paulus, 2000).
While previous studies have explored the impact of manipulating participants’ expectations about active or sham stimulation on treatment efficacy (Braga et al., 2021; Rabipour et al., 2018), the interplay between individual differences in subjective experiences of treatment and assigned intervention conditions (objective treatment) remains unexplored.
Hypothesis and Approach
This study challenges the assumption that successful experimental blinding negates the influence of placebo effects on outcomes. The hypothesis posits that accounting for differences in subjective beliefs can offer insights into the conclusions of previous treatment studies. The study introduces a straightforward approach to analyze existing data and guide future research, emphasizing the importance of considering subjective treatment alongside objective treatment.
Reanalysis of Four Neurostimulation Studies
To test the hypothesis, the study reanalyzes data from four independently published neurostimulation studies, covering clinical and non-clinical samples across different age groups (Blumberger et al., 2016; Filmer et al., 2019; Kaster et al., 2018). The data and analysis codebook are openly accessible on the Open Science Framework (https://osf.io/rztxu/?view_only=879a6dbccd624d87a96cb9d839dd833c).
In this discussion chapter, we delve into the key findings and implications of our study, which employed a novel approach to assess the impact of participants’ subjective beliefs on research outcomes in the field of neurostimulation. The analysis focused on four independent datasets from studies involving repetitive transcranial magnetic stimulation (rTMS) and transcranial direct current stimulation (tDCS) in various populations.
- Subjective Beliefs as a Significant Factor:
- We reveal that participants’ subjective beliefs regarding active versus control (sham) treatment significantly contribute to the variability in primary outcomes.
- In Studies 1, 2, and 4, subjective beliefs explained variability in clinical and cognitive scores to a greater extent than the objective treatment alone.
- The consistency of this pattern across experiments, including the replacement of subjective treatment with subjective dosage in Study 4, underscores the importance of subjective beliefs in understanding research outcomes.
- Causal Role of Subjective Beliefs:
- The question of the causal role of subjective beliefs remains complex and warrants further exploration. Current study designs often record subjective beliefs only at the end of experiments, making it challenging to determine whether changes in performance or symptoms influence beliefs or vice versa.
- In Study 2, where subjective beliefs were queried after the first week, significant results emerged two weeks later, suggesting a potential causal role of beliefs in affecting experimental outcomes.
- Temporal Assessment of Subjective Beliefs:
- Future studies should consider recording subjective beliefs at multiple time points—before, during, and after the experiment—to understand how these beliefs evolve and their differential association with experimental results.
- However, challenges such as increased participant attention to treatment allocation and potential interference with treatment effects need careful consideration.
- Deception as an Alternative Approach:
- An alternative approach involving deception, where all participants are informed they received the active treatment, could mitigate the impact of subjective beliefs on research outcomes.
- While ethical concerns arise, this approach aligns with real-world clinical practices and may enhance the ecological validity of study findings.
Generalizability to Other Treatments:
- The proposed approach is not exclusive to neuromodulation techniques and can be extended to other forms of treatment tested in standard experiments or randomized controlled trials (RCTs). This includes pharmacological studies, cognitive training, and interventions involving virtual reality or brain-computer interfaces.
- The heightened susceptibility of participants to specific expectations in cutting-edge research settings emphasizes the potential intensification of the explanatory power of subjective beliefs.
Objective vs. Subjective Outcomes:
- The impact of subjective beliefs on research outcomes is not limited to self-report measures. The study suggests that placebo effects, influenced by subjective beliefs, may extend to more objective behavioral and neural outcomes.
- Although the study focused on subjective treatment, the interplay between subjective beliefs and participants’ expectations remains unexplored. Future research should consider investigating how expectations modulate the effects of subjective treatment on outcomes.
Limitations and Future Considerations:
- Our study highlights the importance of systematically collecting data on participants’ subjective beliefs and expectations. The consideration of subjective beliefs at different time points, combined with thorough expectations assessment, could enhance the validity and replicability of research results.
- Ethical concerns surrounding deception in clinical populations necessitate careful review, urging researchers to strike a balance between ecological validity and ethical considerations.
In conclusion, our study introduces the concepts of subjective treatment and subjective dosage while shedding light on the substantial role of participants’ subjective experiences in explaining variability in research outcomes. We call for a paradigm shift in research practices, urging future studies to meticulously collect and analyze data on subjective beliefs and expectations. By doing so, we aim to advance the scientific and clinical understanding of treatment effects on human health and behavior.
reference link: https://elifesciences.org/reviewed-preprints/88889#s3