Spin Bias in randomized controlled trials of botulinum toxin for bruxism management: a meta-epidemiologic study

Abstract

Objective

To perform a quantitative and qualitative analysis of spin bias in randomized clinical trials (RCTs) focusing on botulinum toxin (BTX-A) for managing bruxism.

Study design and protocol

This is a meta-epidemiologic study. The protocol was registered on the Open Science Framework.

Study selection

We included RCTs that evaluated the effectiveness of BTX-A for managing bruxism, associated or not with signs and symptoms of temporomandibular disorders. The outcomes were changes in pain and bruxism events. Spin bias was investigated in abstract and main text. The frequency of spin bias was assessed, and a qualitative analysis was conducted. The study was classified as effective if the outcome analyzed was statistically significant (p-value 0.05) and reached the minimum important difference of 20% and ineffective if the reported outcome was statistically nonsignificant or the study did not report the p-value or the results did not reach the minimum important difference of 20%.

Results

An overall frequency of 59.4% spin bias was identified in eight included RCTs. The conclusion in the main text (87.5%) was the section with the highest frequency of spin bias. In the qualitative analysis, the most common strategies identified were inadequate extrapolation to a large population (30.61%), inadequate implication for clinical practice (20.41%), and misleading reporting (12.25%).

Conclusion

There is a high frequency of spin bias in RCTs that evaluated BTX-A for bruxism management. Close to 90% of the selected RCTs presented spin bias in the main text’s conclusion. The most common spin was the inadequate extrapolation of the results.

Clinical Significance

Applying BTX into the temporalis did not reduce muscle activity and the results for masseter injections remain controversial. It seems that BTX-A injections can reduce pain from two weeks to one year. It is not possible to have certainty about the efficacy and safety of using BTX-A to reduce pain and bruxism events.

Introduction

Randomized Controlled Trials (RCTs) are widely considered the gold standard for evaluating the efficacy of health interventions [1]. Various methodologies have been developed to optimize the design, execution, analysis, reporting, and critical evaluation of RCTs [2,3,4]. However, despite rigorous efforts to enhance their methodological integrity, RCTs remain vulnerable to biases that undermine the accurate interpretation of results [5].

The CONSORT guidelines for RCTs emphasize the importance of precise and transparent reporting and interpretation based on the available evidence [6]. Accurate reporting is essential for disseminating knowledge, guiding future research, and informing clinical practice guidelines [6]. However, reporting can be complex, as there is a risk of both conscious and unconscious misinterpretation of results, which may introduce "spin" into the narrative. Spin bias refers to the distortion of result interpretation, potentially misleading readers. This bias can arise from a lack of understanding of the scientific subject, unconscious biases, or even a deliberate intent to mislead [5].

The BMJ introduced the concept of spin bias in biomedical research in 1995 [7]. Boutron et al. [8] initially defined spin bias as specific reporting tactics that, regardless of intent, present an experimental treatment as beneficial despite statistically non-significant primary outcomes or divert attention from such outcomes. More recently, spin bias has been further defined as the intentional or unintentional misrepresentation of study findings [9], with different forms identified, such as emphasizing statistically significant but clinically irrelevant results or interpreting non-significant findings as positive [10]. The prevalence of spin in abstracts has been explored across various fields of medicine [11] and dentistry [12,13,14,15,16,17], as a reminder for readers to critically evaluate new treatment options that may seem too good to be true.

In recent years, growing interest has been shown in the therapeutic potential of Botulinum toxin type A (BTX-A) for bruxism management. This toxin inhibits acetylcholine release from presynaptic nerve endings in striated muscle, causing temporary muscle paralysis [18, 19]. Numerous trials have investigated its effects on pain and bruxism event reduction [20,21,22,23,24,25,26,27]. Given the promising nature of this intervention and the possible commercial interests involved, researchers may be inclined to accentuate its benefits when interpreting RCTs findings. To our knowledge, no study has examined spin in RCTs assessing BTX-A use in bruxism management. Hence, this study aims to identify the frequency of spin bias in such studies and conduct a qualitative analysis of spin bias in publications of RCTs evaluating botulinum toxin use for bruxism management to reduce pain and bruxism events. Also, we aimed to analyze whether the studies are effective from a statistical and clinical point of view.

Methods

Study design and setting

This meta-epidemiological study was conducted at the Universidade Federal de São Paulo, Brazil. The study adhered to a pre-established protocol accessible through the Open Science Framework (OSF) under Doi: 10.17605/OSF.IO/QASC9.

Criteria for including publications

Types of studies

We considered only published RCTs evaluating the effectiveness of BTX-A for managing bruxism, with no restrictions on time or language. To be included, the study must have evaluated pain and/or bruxism events as outcomes. BTX-A could have been applied alongside other therapies (co-interventions) if its effect could be isolated.

Types of participants

We included studies comprising patients diagnosed with bruxism. Bruxism may manifest as an isolated condition or be accompanied by signs and symptoms of temporomandibular disorders. Participants of any age, sex, race, or socioeconomic status were eligible. We accepted studies involving sleep or awake bruxism management. The bruxism could have been identified by different methods: sleep/wake bruxism based solely on positive self-report; probable sleep/wake bruxism based on positive clinical inspection, with or without positive self-report; definitive sleep/wake bruxism based on positive instrumental assessment (polysomnography-PSG or electromyography-EMG), with or without positive self-report and/or positive clinical inspection, adhering to the criteria outlined by Lobbezoo et al. (2018) [28].

Types of interventions

We considered any form of BTX-A used for therapeutic purposes for bruxism. The drug should have been administered into the masseters and/or temporalis muscles, employing various administration protocols and follow-up periods.

Types of comparators

The effects of the intervention should have been assessed through any of the following comparisons: versus sham therapy or placebo; no treatment; other interventions alone (e.g., occlusal splints, medication use); or BTX-A plus other interventions versus other interventions alone.

Types of outcomes

We considered only RCTs in which the outcomes were in pain and/or bruxism events. The pain must have been measured using a Visual Analog Scale (VAS) or another validated instrument. In case more than one scale was used, the results from the VAS were prioritized for the analysis to improve the consistency and comparability of our analysis. The bruxism events must have been measured by EMG or PSG and expressed as masticatory muscle activity or number of bruxism episodes per hour.

Exclusion criteria

• 1. Study types other than published RCTs (observational studies, non-randomized clinical trials, pilot studies, and study protocols).

• 2. Studies that do not present the outcome of interest.

• 3. Studies in which the sample was composed only of people presenting congenital abnormalities, cerebral palsy, orofacial or craniocervical dystonia, Parkinson’s disease, autism, etc.

• 4. Studies involving patients with secondary bruxism (caused by or associated with neurological disorders or medication use).

• 5. Studies in which BTX-A was applied for another purpose than bruxism.

• 6. Studies that analyzed BTX-A in different administration doses in both groups and/or studies without a control group that did not receive BTX-A.

Identification and selection of publications

The Cochrane Highly Sensitive Search Strategy [29] was used in the string strategy on MEDLINE (via PubMed) to identify RCTs that evaluated BTX-A for bruxism. We applied this string, and one author (GDC) read the titles, abstracts, and keywords of the identified references to collect terms to improve the search strategy. Afterward, with the assistance of a Health Science Librarian, appropriate word combinations were adapted in new search strategies for each electronic database: MEDLINE (via PubMed), Cochrane (Central), Embase, and LILACS (via BVS), from inception to June 10th, 2024, when the strategies were applied. No language or publication date restrictions were set. The final strategy for each database is available in Appendix A.

Study selection process

The reference files of each database were imported into a reference software manager (EndNote X9®; Thomson Reuters, Philadelphia, PA, USA). EndNote initially managed duplicate removal, which was then manually reviewed using Rayyan® Online Software (Qatar Computing Research Institute, Qatar).

The Rayyan® Online Software (Qatar Computing Research Institute, Qatar) was used throughout the study selection process. Two authors (GDC; PP) were calibrated by reading ten abstracts, according to the eligibility criteria, before the selection process. Then, these authors (GDC; PP) participated in the study selection process. If any disagreements arise, the third author (TMSVG) was involved in reaching a final decision. The selection process was conducted in two phases. During phase 1, the identified RCTs underwent an initial screening process based on their titles and abstracts. In phase 2, RCTs that passed the initial screening stage underwent a full-text assessment.

Data extraction

One author (GDC) collected data from the selected articles using standardized spreadsheets previously prepared by the authors. A second author (TMSVG) cross-checked all the data.

Spin bias analysis

Quantitative analyses of spin bias (Frequency)

Spin bias was investigated in four sections of the publication: the abstract results, abstract conclusions, main text results, and main text conclusions.

Based on the concept of spin bias, “the intentional or unintentional misrepresentation of study findings [9],” two independent authors (GDC and TMSVG) classified the sections with spin (yes) or without (no) to identify spin bias. After that, the two authors (GDC and TMSVG) held a consensus meeting to decide on the presence or absence of spin in each section. Then, we determined the spin frequency in each section and across the sample of the included studies.

The proportion of spin in each section was calculated considering the number of spin × 100, divided by the total of RCTs included in the analysis. The proportion of spin in all studies was calculated considering the sum of spins × 100, divided by the total of RCTs included in the analysis.

Qualitative analysis

A qualitative analysis of spin bias was conducted by quoting sentences related to the identified spin bias and offering plausible explanations for any disagreements with the original wording. If spin bias was detected, it was classified by two authors (GDC and TMVG) into categories as recommended by Lazarus [30] (Appendix B). A third author (CMS) crosscheck the data. Descriptive statistics were used to summarize the frequency and characteristics of the spin, and they were categorized into three types: (1) misleading reporting, (2) inadequate interpretation, and (3) inadequate extrapolation [30].

Statistical analysis and clinical applicability

We tabulated the p-values for the differences between the groups described in each article. This data was also used to calculate the change in mean values after the intervention (across different follow-up periods) compared to baseline, with the differences expressed as percentages. We defined the minimum important difference (MID) as the smallest meaningful change in the scores of both outcomes. For our study, we defined the Minimum Important Difference (MID) as a 20% improvement for both pain and bruxism events. Given the absence of established MID values for bruxism events in the literature, we based our decision on Calixtre et al [31]. Calixtre et al [31]. found that the MID for general chronic pain, measured using the Visual Analog Scale (VAS), ranges from 1.5 to 3.2 cm, and a pain reduction of 3.2 cm on the VAS may not be necessary to demonstrate clinically meaningful improvement in TMD patients [31].

Based on this threshold, we classified the study results as either effective or ineffective. The study was classified as effective if the analyzed outcome was statistically significant (p-value < 0.05) and achieved a 20% improvement (MID). It was classified as ineffective if the reported outcome was statistically non-significant, the p-value was not provided, or the results did not reach the 20% improvement threshold.

Results

Study selection

We identified 211 references, of which 125 remained after duplicate removal to begin phase 1. Based on the eligibility criteria, 110 references were excluded, leaving 15 studies for the full-text analysis in phase 2. At the end of phase 2, eight articles were included in the final analysis. A flowchart outlining the study selection process is presented in Fig. 1. The excluded references, along with the reasons for their exclusion, are provided in Appendix C.

Fig. 1

Flow diagram

Study characteristics

The studies were published between 2010 [22] and 2024 [23], with sample sizes ranging from 12 [22] to 73 patients [23], totaling 244 participants. Six studies were conducted in Asia [20,21,22,23, 25, 26], one in the Americas [24], and one in Turkey [27]. The main characteristics of the included studies are summarized in Table 1.

Table 1 Main characteristics of included studies (n = 8)

Most of the studies focused on sleep bruxism. One study [23] included patients with both awake and sleep bruxism, while in another, this information was unclear [27]. Bruxism diagnosis was made using a variety of methods, including questionnaires [22], questionnaires combined with clinical assessment [20, 21, 23, 25, 26], or EMG/PSG [24]. One study relied solely on clinical assessment to identify bruxism patients [27].

The BTX-A was injected bilaterally in the masseter muscle [20, 22, 23] or in the masseter and temporalis [21, 24, 27].

Of the eight included studies, only two [25, 27] reported using the CONSORT guidelines [6], and just four studies provided a flow diagram outlining the participant selection process [23, 25,26,27]. Two studies were funded by the pharmaceutical industry [24, 26]. Only two studies included a sample size calculation [23, 25]. Three studies used a random number generator program for randomization [24, 25, 27], while one used the slot method [23]. The bibliometric characteristics of the included studies are detailed in Table 2 and Appendix D.

Table 2 Summary of bibliometric characteristics of the included studies

Spin bias analysis

Quantitative analyses of spin bias (Frequency)

We identified spin bias in all of the included articles. Across the four sections analyzed in the eight RCTs, 59.4% exhibited spin bias. When considered individually, a high frequency of spin bias was observed in each section. The section with the highest frequency of spin bias was the conclusion in the main text, which occurred in 87.5% of the studies (Fig. 2/Appendix E).

Fig. 2

Frequency of spin bias in the included studies

Qualitative analysis

The most common spin strategies identified were inadequate extrapolation to a larger population (30.61%), inadequate implications for clinical practice (20.41%), and selective reporting (12.25%). Table 3 presents the qualitative analysis of the spin bias and the classification of the identified spin types. Table 4 shows the frequency of each spin classification, while Fig. 3 illustrates the most frequent forms of spin.

Table 3 Qualitative and categorical analysis of included publications where spin bias was detected

Table 4 Categorization of Spin according to Lazarus [30]

Fig. 3

Types of spin and frequency

Clinical and Statistical Effectiveness

Pain (Fig. 4 / Appendix F):

Fig. 4

Results for pain outcome in different follow-ups (n = 4)

Five studies evaluated pain [20, 21, 23, 25, 27], considering different types of pain: pain at rest and during chewing, pain upon palpation of the chewing muscles, perceived pain, and unspecified pain types.

The results for BTX-A application were statistically significant and led to a reduction of more than 20% in pain, meeting the MID at various time points: two weeks [23, 25], one month [25], two months [20], three months [25], four months [25], six months [20, 25, 27], and one year [20]. However, Al-Wayli (2017) [20] did not show effectiveness for three weeks. The studies by Al-Wayli (2024) [23] and Jadhao et al. (2017) [21] were classified as non-effective because they did not report the p-value.

Among these five studies, two exhibited 25% spin bias [21, 25], one showed 50% spin bias [27], and two had 100% spin bias [20, 23], meaning spin bias was present in all four sections analyzed for these studies.

Bruxism (Fig. 5/ Appendix G):

Fig. 5

Results for bruxism events in different follow-ups

Four studies evaluated bruxism events using different measures: muscular activity (microvolts) [25], the number of bruxism events [22, 24], and rhythmic masticatory muscle activity (RMMA) episodes per hour [26].

The results of BTX-A application to the masseter were statistically significant, reducing muscle activity by more than 20%, and meeting the MID at two weeks [25] and two months [22]. The results were inconsistent at one and three months, and by six months, no effects were observed [25].

On the other hand, applying BTX-A to the temporalis muscle was ineffective and did not reduce muscle activity [22], regardless of the follow-up period.

Among these four studies, two exhibited 25% spin bias [25, 26], one showed 50% spin bias [22], and one had 100% spin bias [24].

Discussion

This study aimed to identify the frequency of spin bias in RCTs evaluating BTX-A use for bruxism management and conduct a qualitative analysis of the spin bias present in these publications. We found that 100% of the included RCTs exhibited spin bias in at least one section analyzed. The frequency of spin bias was particularly high in the conclusions of the main text.

Numerous studies have examined spin bias in dentistry, revealing a high frequency of spin bias in the dental literature [12,13,14,15,16,17]. The prevalence of spin bias in dental articles' abstracts has been reported to range from 31% [16] to 60% [12]. Spin bias has been observed across various dental specialties, with 62.2% in orthodontics [14], 69% in periodontology and oral implantology [17], 78.4% in pediatric dentistry [15], 79% in dental caries [32], and 85% in endodontics [13]. In the field of sleep medicine, Guo et al. (2023) [15] evaluated the prevalence and characteristics of spin in RCT abstracts, finding that 78.1% contained at least one form of spin bias, with 57.9% exhibiting spin in the results section and 71.9% in the conclusions section. These findings align with the results of our study, which identified approximately 59% of spin bias across the included studies.

Our comprehensive literature search identified only eight RCTs that compared patients who received BTX-A with those who did not (control, sham, placebo, or other treatment) for bruxism management. These studies included a total of 244 patients, with 143 receiving BTX-A. The studies were highly heterogeneous, which made it challenging to synthesize results and draw meaningful comparisons. Despite several attempts to group the findings in systematic reviews [33,34,35,36,37,38,39,40,41,42,43,44,45], only three of these reviews conducted a meta-analysis to synthesize the results [34, 35, 44]. Recently, Coelho et al. [46] published an overview protocol, highlighting that these systematic reviews yield controversial results, underscoring the need for further clarification.

For over 100 years, the results of scientific research have been primarily analyzed by focusing on the p-value, using an arbitrary threshold of < 0.05 as a "magic" cutoff to determine statistical significance [47]. A new concept has recently emerged in the literature that encourages a more critical analysis of clinical study results. This approach emphasizes that, beyond statistical significance, an intervention must also be clinically meaningful for patients. Schünemann and Guyatt (2005) [48] defined the Minimum Important Difference (MID) as the smallest difference in an outcome that patients or their proxies perceive as important, either beneficial or harmful, and which would lead to a change in clinical management. Therefore, the interpretation of research results should go beyond just the numbers. As illustrated by the qualitative analysis in our study, many authors [20,21,22, 26, 27, 49] tend to exaggerate the significance of their findings, offering overly assertive conclusions that, in some cases, are not based on actual data but on assumptions [8]. This highlights the importance of critically evaluating both statistical and clinical relevance in research.

The qualitative analysis in our study revealed that more than half of the identified spin bias involved the inadequate extrapolation of results—either to a larger population or to clinical practice. This type of spin can mislead inattentive readers, potentially prompting them to apply ineffective therapies based on a cursory reading of only the abstracts of scientific articles.

In this context, we must consider the importance of the linguistic perspective [7]. Authors should be trained to report only what is directly supported by the results, avoiding strategies that exaggerate or extrapolate the findings [7]. Similarly, readers should be equipped with the necessary tools to identify the author's intentions in their writing [7]. When spin bias goes unnoticed by readers, it can lead to clinical misconduct by adopting ineffective or unproven interventions [50]. To reduce spin bias, training journal reviewers to detect it is crucial [8].

For clinicians, it is important to recognize that although BTX-A has been used for bruxism treatment for over 15 years [22], its efficacy and safety remain unproven. The articles included in our study have significant limitations: only two RCTs reported sample size calculations [25, 48]; only two [25, 27] adhered to CONSORT guidelines [6], and two studies were funded by the pharmaceutical industry [24, 26], raising potential conflicts of interest. Of the eight studies, only five evaluated adverse effects [20, 22, 24, 25, 27], with three reporting no adverse effects [20, 22, 27], one mentioning cosmetic changes in two patients [22], and another citing pain and discomfort at the injection sites in some patients [25]. Additionally, all the RCTs focused on young adults, primarily in Asia, which limits the generalizability of the results to global clinical practice. These limitations reduce the study’s external validity, as bruxism was assessed using varying eligibility criteria and detection methods.

Regarding bruxism events, the results showed that BTX-A had no lasting effect after six months [25], and injections into the temporalis did not reduce muscle activity [22]. The results for masseter injections were inconsistent, based on only four studies [22, 24,25,26], which collectively analyzed just 76 patients.

In terms of pain, there is a common belief that BTX-A reduces pain. However, these conclusions are based on just four studies, with a total sample of 183 patients [20, 25, 27, 49]. In addition to the small sample size, it is important to note that only one of these studies exhibited less than 50% spin bias [25]. Furthermore, these RCTs had a significant conceptual flaw. Most of the studies evaluated pain and bruxism events as outcomes, yet bruxism itself does not directly cause pain. Bruxism is a central-origin activity characterized by repetitive jaw-muscle movements, including clenching or grinding of the teeth, and/or bracing or thrusting of the mandible [51, 52]. While bruxism can be correlated with painful temporomandibular disorder (TMD) [53], a definitive causal relationship between bruxism and TMD has yet to be conclusively established [54]. The prevalence of spin in studies of botulinum toxin A (BTX-A) for bruxism suggests potential weaknesses in the design and execution of these clinical trials. These methodological shortcomings may contribute to difficulties in conducting the research and subsequently complicate the reporting of findings.

The main strength of this study lies in the robust methodology employed, with the selection and analysis process carried out by two independent evaluators, reducing the potential for bias. However, there are some limitations. One is the subjectivity inherent in qualitative analysis. Additionally, our research does not allow us to determine the underlying reasons for the spin biases identified—whether they were intentional, a lack of knowledge, or a combination of both. The small number of included studies also limited our ability to conduct more sophisticated quantitative analyses.

Clear, transparent, and objective reporting of research results is crucial for ensuring the accurate reflection of data and preventing misinterpretations that could negatively impact clinical practice. Inadequate reporting can lead to poor clinical decisions and potentially harm patients. Our study revealed spin bias in 87.5% of main text conclusions and 62.5% of abstract conclusions, a finding particularly concerning given that clinicians may rely solely on these conclusions when making treatment decisions. To mitigate this risk, clinicians should receive training in critically appraising published research, including education on study design, potential biases, and conflicts of interest. This is especially important in therapeutic areas with substantial industry funding, such as botulinum toxin therapies.

Given the current state of the literature, more RCTs are needed to establish the efficacy and safety of BTX-A for reducing pain and bruxism events, as there is insufficient evidence to draw definitive conclusions.

Conclusion

This study suggests that spin bias is highly prevalent in RCTs evaluating BTX-A for bruxism management. Specifically, nearly 90% of the reviewed RCTs showed spin bias in their conclusions, with excessive extrapolation of results being the most common issue.