Quality ratings of reviews in overviews: a comparison of reviews with and without dual (co-)authorship

Abstract Background Previous research shows that many authors of Cochrane overviews were also involved in some of the included systematic reviews (SRs). This type of dual (co-)authorship (DCA) may be a conflict of interest and a potential source of bias. Our objectives were to (1) additionally investigate DCA in non-Cochrane overviews; (2) investigate whether there is an association between DCA and quality assessments of SRs in Cochrane and non-Cochrane overviews. Methods We selected a sample of Cochrane (n = 20) and non-Cochrane (n = 78) overviews for analysis. We extracted data on the number of reviews affected by DCA and whether quality assessment of included reviews was conducted independently. Differences in mean quality scores between SRs with and without DCA were calculated in each overview. These differences were standardized (using the standardized mean difference (SMD)) and meta-analyzed using a random effects model. Results Forty out of 78 non-Cochrane overviews (51%) and 18 out of 20 Cochrane overviews (90%) had included at least one SR with DCA. For Cochrane overviews, a median of 5 [interquartile range (IQR) 2.5 to 7] SRs were affected by DCA (median of included reviews 10). For non-Cochrane overviews a median of 1 [IQR 0 to 2] of the included SRs were affected (median of included reviews 14). The meta-analysis showed a SMD of 0.58 (95% confidence interval (CI) 0.27 to 0.90) indicating higher quality scores in reviews with overlapping authors. The test for subgroup differences shows no evidence of a difference between Cochrane (SMD 0.44; 95% CI 0.07 to 0.81) and non-Cochrane overviews (SMD 0.62; 95% CI 0.06 to 1.17). Conclusions Many authors of overviews also often have an authorship on one or more of the underlying reviews. Our analysis shows that, on average, authors of overviews give higher quality ratings to SRs in which they were involved themselves than to other SRs. Conflict of interest is one explanation, but there are several others such as reviewer expertise. Independent and blinded reassessments of the reviews would provide more robust evidence on potential bias arising from DCA

Communicating Uncertainty in Written Consumer Health Information to the Public: Parallel-Group, Web-Based Randomized Controlled Trial

Background Uncertainty is integral to evidence-informed decision making and is of particular importance for preference-sensitive decisions. Communicating uncertainty to patients and the public has long been identified as a goal in the informed and shared decision-making movement. Despite this, there is little quantitative research on how uncertainty in health information is perceived by readers. Objective The aim of this study was to examine the impact of different uncertainty descriptions regarding the evidence for a treatment effect in a written research summary for the public. Methods We developed 8 versions of a research summary on a fictitious drug for tinnitus with varying degrees (Q1), sources (Q2), and magnitudes of uncertainty (Q3). We recruited 2099 members of the German public from a web-based research panel. Of these, 1727 fulfilled the inclusion criteria and were randomly presented with one of these research summaries. Randomization was conducted by using a centralized computer with a random number generator. Web-based recruitment and data collection were fully automated. Participants were not aware of the purpose of the study and alternative presentations. We measured the following outcomes: perception of the treatment effectiveness (primary), certainty in the judgement of treatment effectiveness, perception of the body of evidence, text quality, and intended decision. The outcomes were self-assessed. Results For the primary outcome, we did not find a global effect for Q1 and Q2 (P=.25 and P=.73), but we found a global effect for Q3 (P=.048). Pairwise comparisons showed a weaker perception of treatment effectiveness for the research summary with 3 sources of uncertainty compared to the version with 2 sources of uncertainty (P=.04). Specifically, the proportion of the participants in the group with 3 sources of uncertainty that perceived the drug as possibly beneficial was 9% lower than that of the participants in the group with 2 sources of uncertainty (92/195, 47.2% vs 111/197, 56.3%, respectively). The proportion of the participants in the group with 3 sources of uncertainty that considered the drug to be of unclear benefit was 8% higher than that of the participants in the group with 2 sources of uncertainty (72/195, 36.9% vs 57/197, 28.9%, respectively). However, there was no significant difference compared to the version with 1 source of uncertainty (P=.31). We did not find any meaningful differences between the research summaries for the secondary outcomes. Conclusions Communicating even a large magnitude of uncertainty for a treatment effect had little impact on the perceived effectiveness. Efforts to improve public understanding of research are needed to improve the understanding of evidence-based health information. Trial Registration German Clinical Trials Register DRKS00015911, https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&amp;TRIAL_ID=DRKS00015911 International Registered Report Identifier (IRRID) RR2-10.2196/13425 </jats:sec

Communicating Uncertainty in Written Consumer Health Information to the Public: Parallel-Group, Web-Based Randomized Controlled Trial (Preprint)

BACKGROUND Uncertainty is integral to evidence-informed decision making and is of particular importance for preference-sensitive decisions. Communicating uncertainty to patients and the public has long been identified as a goal in the informed and shared decision-making movement. Despite this, there is little quantitative research on how uncertainty in health information is perceived by readers. OBJECTIVE The aim of this study was to examine the impact of different uncertainty descriptions regarding the evidence for a treatment effect in a written research summary for the public. METHODS We developed 8 versions of a research summary on a fictitious drug for tinnitus with varying degrees (Q1), sources (Q2), and magnitudes of uncertainty (Q3). We recruited 2099 members of the German public from a web-based research panel. Of these, 1727 fulfilled the inclusion criteria and were randomly presented with one of these research summaries. Randomization was conducted by using a centralized computer with a random number generator. Web-based recruitment and data collection were fully automated. Participants were not aware of the purpose of the study and alternative presentations. We measured the following outcomes: perception of the treatment effectiveness (primary), certainty in the judgement of treatment effectiveness, perception of the body of evidence, text quality, and intended decision. The outcomes were self-assessed. RESULTS For the primary outcome, we did not find a global effect for Q1 and Q2 (&lt;i&gt;P&lt;/i&gt;=.25 and &lt;i&gt;P&lt;/i&gt;=.73), but we found a global effect for Q3 (&lt;i&gt;P&lt;/i&gt;=.048). Pairwise comparisons showed a weaker perception of treatment effectiveness for the research summary with 3 sources of uncertainty compared to the version with 2 sources of uncertainty (&lt;i&gt;P&lt;/i&gt;=.04). Specifically, the proportion of the participants in the group with 3 sources of uncertainty that perceived the drug as possibly beneficial was 9% lower than that of the participants in the group with 2 sources of uncertainty (92/195, 47.2% vs 111/197, 56.3%, respectively). The proportion of the participants in the group with 3 sources of uncertainty that considered the drug to be of unclear benefit was 8% higher than that of the participants in the group with 2 sources of uncertainty (72/195, 36.9% vs 57/197, 28.9%, respectively). However, there was no significant difference compared to the version with 1 source of uncertainty (&lt;i&gt;P&lt;/i&gt;=.31). We did not find any meaningful differences between the research summaries for the secondary outcomes. CONCLUSIONS Communicating even a large magnitude of uncertainty for a treatment effect had little impact on the perceived effectiveness. Efforts to improve public understanding of research are needed to improve the understanding of evidence-based health information. CLINICALTRIAL German Clinical Trials Register DRKS00015911, https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&amp;amp;TRIAL_ID=DRKS00015911 INTERNATIONAL REGISTERED REPORT RR2-10.2196/13425 </sec