Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study

Background Empirical research has illustrated an association between study size and relative treatment effects, but conclusions have been inconsistent about the association of study size with the risk of bias items. Small studies give generally imprecisely estimated treatment effects, and study variance can serve as a surrogate for study size. Methods We conducted a network meta-epidemiological study analyzing 32 networks including 613 randomized controlled trials, and used Bayesian network meta-analysis and meta-regression models to evaluate the impact of trial characteristics and study variance on the results of network meta-analysis. We examined changes in relative effects and between-studies variation in network meta-regression models as a function of the variance of the observed effect size and indicators for the adequacy of each risk of bias item. Adjustment was performed both within and across networks, allowing for between-networks variability. Results Imprecise studies with large variances tended to exaggerate the effects of the active or new intervention in the majority of networks, with a ratio of odds ratios of 1.83 (95% CI: 1.09,3.32). Inappropriate or unclear conduct of random sequence generation and allocation concealment, as well as lack of blinding of patients and outcome assessors, did not materially impact on the summary results. Imprecise studies also appeared to be more prone to inadequate conduct. Conclusions Compared to more precise studies, studies with large variance may give substantially different answers that alter the results of network meta-analyses for dichotomous outcome

What guidance are researchers given on how to present network meta-analyses to end-users such as policymakers and clinicians? A systematic review

© 2014 Sullivan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Introduction: Network meta-analyses (NMAs) are complex methodological approaches that may be challenging for non-technical end-users, such as policymakers and clinicians, to understand. Consideration should be given to identifying optimal approaches to presenting NMAs that help clarify analyses. It is unclear what guidance researchers currently have on how to present and tailor NMAs to different end-users. Methods: A systematic review of NMA guidelines was conducted to identify guidance on how to present NMAs. Electronic databases and supplementary sources were searched for NMA guidelines. Presentation format details related to sample formats, target audiences, data sources, analysis methods and results were extracted and frequencies tabulated. Guideline quality was assessed following criteria developed for clinical practice guidelines. Results: Seven guidelines were included. Current guidelines focus on how to conduct NMAs but provide limited guidance to researchers on how to best present analyses to different end-users. None of the guidelines provided reporting templates. Few guidelines provided advice on tailoring presentations to different end-users, such as policymakers. Available guidance on presentation formats focused on evidence networks, characteristics of individual trials, comparisons between direct and indirect estimates and assumptions of heterogeneity and/or inconsistency. Some guidelines also provided examples of figures and tables that could be used to present information. Conclusions: Limited guidance exists for researchers on how best to present NMAs in an accessible format, especially for non-technical end-users such as policymakers and clinicians. NMA guidelines may require further integration with end-users' needs, when NMAs are used to support healthcare policy and practice decisions. Developing presentation formats that enhance understanding and accessibility of NMAs could also enhance the transparency and legitimacy of decisions informed by NMAs.The Canadian Institute of Health Research (CIHR) Drug Safety and Effectiveness Network (Funding reference number – 116573)

Conservative interventions for urinary incontinence in women: an overview of Cochrane systematic reviews

This is the protocol for a review and there is no abstract. The objectives are as follows:  To synthesise Cochrane reviews of conservative interventions, as described above, for the prevention or treatment of female urinary incontinence.Output Type: Protoco

A Microsoft-Excel-based tool for running and critically appraising network meta-analyses--an overview and application of NetMetaXL.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.BACKGROUND: The use of network meta-analysis has increased dramatically in recent years. WinBUGS, a freely available Bayesian software package, has been the most widely used software package to conduct network meta-analyses. However, the learning curve for WinBUGS can be daunting, especially for new users. Furthermore, critical appraisal of network meta-analyses conducted in WinBUGS can be challenging given its limited data manipulation capabilities and the fact that generation of graphical output from network meta-analyses often relies on different software packages than the analyses themselves. METHODS: We developed a freely available Microsoft-Excel-based tool called NetMetaXL, programmed in Visual Basic for Applications, which provides an interface for conducting a Bayesian network meta-analysis using WinBUGS from within Microsoft Excel. . This tool allows the user to easily prepare and enter data, set model assumptions, and run the network meta-analysis, with results being automatically displayed in an Excel spreadsheet. It also contains macros that use NetMetaXL's interface to generate evidence network diagrams, forest plots, league tables of pairwise comparisons, probability plots (rankograms), and inconsistency plots within Microsoft Excel. All figures generated are publication quality, thereby increasing the efficiency of knowledge transfer and manuscript preparation. RESULTS: We demonstrate the application of NetMetaXL using data from a network meta-analysis published previously which compares combined resynchronization and implantable defibrillator therapy in left ventricular dysfunction. We replicate results from the previous publication while demonstrating result summaries generated by the software. CONCLUSIONS: Use of the freely available NetMetaXL successfully demonstrated its ability to make running network meta-analyses more accessible to novice WinBUGS users by allowing analyses to be conducted entirely within Microsoft Excel. NetMetaXL also allows for more efficient and transparent critical appraisal of network meta-analyses, enhanced standardization of reporting, and integration with health economic evaluations which are frequently Excel-based.CC is a recipient of a Vanier Canada Graduate Scholarship from the Canadian Institutes of Health Research (funding reference number—CGV 121171) and is a trainee on the Canadian Institutes of Health Research Drug Safety and Effectiveness Network team grant (funding reference number—116573). BH is funded by a New Investigator award from the Canadian Institutes of Health Research and the Drug Safety and Effectiveness Network. This research was partly supported by funding from CADTH as part of a project to develop Excel-based tools to support the conduct of health technology assessments. This research was also supported by Cornerstone Research Group

Interventions for treating cavitated or dentine carious lesions

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows: To determine the clinical and cost-effectiveness of interventions (non-selective, selective or stepwise carious tissue removal, sealing of carious lesions using sealant materials or preformed metal crowns, or NRCC) to treat carious lesions conventionally considered to require restorations (cavitated or micro-cavitated lesions, or occlusal lesions that are clinically non-cavitated but clinically/radiographically extend into dentine) in primary or permanent teeth with vital (sensitive) pulps.</p

A novel approach for identifying and addressing case‐mix heterogeneity in individual participant data meta‐analysis

Case-mix heterogeneity across studies complicates meta-analyses. As a result of this, treatments that are equally effective on patient subgroups may appear to have different effectiveness on patient populations with different case mix. It is therefore important that meta-analyses be explicit for what patient population they describe the treatment effect. To achieve this, we develop a new approach for meta-analysis of randomized clinical trials, which use individual patient data (IPD) from all trials to infer the treatment effect for the patient population in a given trial, based on direct standardization using either outcome regression (OCR) or inverse probability weighting (IPW). Accompanying random-effect meta-analysis models are developed. The new approach enables disentangling heterogeneity due to case mix from that due to beyond case-mix reasons

Producing treatment hierarchies in network meta-analysis using probabilistic models and treatment-choice criteria

A key output of network meta-analysis (NMA) is the relative ranking of the treatments; nevertheless, it has attracted a lot of criticism. This is mainly due to the fact that ranking is an influential output and prone to over-interpretations even when relative effects imply small differences between treatments. To date, common ranking methods rely on metrics that lack a straightforward interpretation, while it is still unclear how to measure their uncertainty. We introduce a novel framework for estimating treatment hierarchies in NMA. At first, we formulate a mathematical expression that defines a treatment choice criterion (TCC) based on clinically important values. This TCC is applied to the study treatment effects to generate paired data indicating treatment preferences or ties. Then, we synthesize the paired data across studies using an extension of the so-called "Bradley-Terry" model. We assign to each treatment a latent variable interpreted as the treatment "ability" and we estimate the ability parameters within a regression model. Higher ability estimates correspond to higher positions in the final ranking. We further extend our model to adjust for covariates that may affect treatment selection. We illustrate the proposed approach and compare it with alternatives in two datasets: a network comparing 18 antidepressants for major depression and a network comparing 6 antihypertensives for the incidence of diabetes. Our approach provides a robust and interpretable treatment hierarchy which accounts for clinically important values and is presented alongside with uncertainty measures. Overall, the proposed framework offers a novel approach for ranking in NMA based on concrete criteria and preserves from over-interpretation of unimportant differences between treatments

Systematic review of interventions for treating or preventing antipsychotic-induced tardive dyskinesia

Background: Antipsychotic medication can cause tardive dyskinesia (TD) – late-onset, involuntary, repetitive movements, often involving the face and tongue. TD occurs in > 20% of adults taking antipsychotic medication (first-generation antipsychotics for > 3 months), with this proportion increasing by 5% per year among those who continue to use these drugs. The incidence of TD among those taking newer antipsychotics is not different from the rate in people who have used older-generation drugs in moderate doses. Studies of TD have previously been found to be limited, with no treatment approach shown to be effective. Objectives: To summarise the clinical effectiveness and safety of treatments for TD by updating past Cochrane reviews with new evidence and improved methods; to undertake public consultation to gauge the importance of the topic for people living with TD/the risk of TD; and to make available all data from relevant trials. Data sources: All relevant randomised controlled trials (RCTs) and observational studies. Review methods: Cochrane review methods, network meta-analysis (NMA). Design: Systematic reviews, patient and public involvement consultation and NMA. Setting: Any setting, inpatient or outpatient. Participants: For systematic reviews, adults with TD who have been taking a stable antipsychotic drug dose for > 3 months. Interventions: Any, with emphasis on those relevant to UK NHS practice. Main outcome measures: Any measure of TD, global assessments and adverse effects/events. Results: We included 112 studies (nine Cochrane reviews). Overall, risk of bias showed little sign of improvement over two decades. Taking the outcome of ‘TD symptoms improved to a clinically important extent’, we identified two trials investigating reduction of antipsychotic dose [n = 17, risk ratio (RR) 0.42, 95% confidence interval (CI) 0.17 to 1.04; very low quality]. Switching was investigated twice in trials that could not be combined (switching to risperidone vs. antipsychotic withdrawal: one RCT, n = 42, RR 0.45, 95% CI 0.23 to 0.89; low quality; switching to quetiapine vs. haloperidol: one RCT, n = 45, RR 0.80, 95% CI 0.52 to 1.22; low quality). In addition to RCTs, six observational studies compared antipsychotic discontinuation with decreased or increased dosage, and there was no clear evidence that any of these strategies had a beneficial effect on TD symptoms (very low-quality evidence). We evaluated the addition to standard antipsychotic care of several treatments, but not anticholinergic treatments, for which we identified no trials. We found no clear effect of the addition of either benzodiazepines (two RCTs, n = 32, RR 1.12, 95% CI 0.6 to 2.09; very low quality) or vitamin E (six RCTs, n = 264, RR 0.95, 95% CI 0.89 to 1.01; low quality). Buspirone as an adjunctive treatment did have some effect in one small study (n = 42, RR 0.53, 95% CI 0.33 to 0.84; low quality), as did hypnosis and relaxation (one RCT, n = 15, RR 0.45, 95% CI 0.21 to 0.94; very low quality). We identified no studies focusing on TD in people with dementia. The NMA model found indirect estimates to be imprecise and failed to produce useful summaries on relative effects of interventions or interpretable results for decision-making. Consultation with people with/at risk of TD highlighted that management of TD remains a concern, and found that people are deeply disappointed at the length of time it has taken researchers to address the issue. Limitations: Most studies remain small and poorly reported. Conclusions: Clinicians, policy-makers and people with/at risk of TD are little better informed than they were decades ago. Underpowered trials of limited quality repeatedly fail to provide answers. Future work: TD reviews have data from current trials extracted, tabulated and traceable to source. The NMA highlights one context in which support for this technique is ill advised. All relevant trials, even if not primarily addressing the issue of TD, should report appropriate binary outcomes on groups of people with this problem. Randomised trials of treatments for people with established TD are indicated. These should be large (> 800 participants), necessitating accrual through accurate local/national registers, including an intervention with acceptable treatments and recording outcomes used in clinical practice. Study registration: This study is registered as PROSPERO CRD4201502045. Funding: The National Institute for Health Research Health Technology Assessment programme

Network meta-analyses should be the highest level of evidence in treatment guidelines

(Editorial

Evaluating the quality of evidence from a network meta-analysis

Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses. © 2014 Salanti et al