Validating the Role of Chatgpt in Automatic Systematic Reviews and Meta-Analysis: a Case Study Focusing on Mortality Benefits of Glucagon-Like Peptide-1 Receptor Agonists

Background Glucagon-like peptide-1 receptor agonists (GLP-1 RAs) play a crucial role in managing type 2 diabetes mellitus (T2DM) and obesity, providing benefits such as glycemic control, weight loss, and cardiovascular protection. Evaluating the impact of GLP-1 RAs on all-cause mortality and cardiovascular mortality is vital for understanding their broader health benefits. This study aims to leverage AI tools, specifically ChatGPT, to enhance the efficiency and accuracy of systematic reviews and meta-analyses, thus addressing methodological limitations and heterogeneities seen in traditional approaches. Objective To determine the effect of GLP-1 receptor agonists on all-cause and cardiovascular mortality in patients with type 2 diabetes and obesity through a comprehensive systematic review and meta-analysis, utilizing AI for improved methodological efficiency. Eligibility Criteria • Inclusion Criteria: o Placebo-controlled randomized controlled trials (RCTs). o Studies reporting mortality outcomes (all-cause and cardiovascular). o Studies involving adult participants receiving GLP-1 RAs. • Exclusion Criteria: o Studies with active comparators instead of placebo controls. o Post hoc analyses and duplicate publications. o Studies not reporting relevant mortality data. Search Strategy • A comprehensive literature search will be conducted using PubMed and Embase databases. • Search terms will include specific GLP-1 RAs (e.g., exenatide, liraglutide, semaglutide) and synonyms for mortality outcomes. • AI tools (ChatGPT) will be employed to generate and refine search strings, validated by human reviewers to ensure accuracy and comprehensiveness. • Example Search Strings: PubMed: ("Exenatide"[MeSH Terms] OR "Exenatide"[Title/Abstract] OR "Liraglutide"[MeSH Terms] OR "Liraglutide"[Title/Abstract] OR "semaglutide"[Title/Abstract] OR "dulaglutide"[Title/Abstract] OR "albiglutide"[Title/Abstract] OR "lixisenatide"[Title/Abstract]) AND ("Mortality"[MeSH Terms] OR "mortalit*"[Title/Abstract] OR "death"[MeSH Terms] OR "death*"[Title/Abstract] OR "survival"[MeSH Terms] OR "survival"[Title/Abstract]) Embase: ('exenatide'/exp OR 'exenatide' OR 'liraglutide'/exp OR 'liraglutide' OR 'semaglutide' OR 'dulaglutide' OR 'albiglutide' OR 'lixisenatide') AND ('mortality'/exp OR mortalit* OR 'death'/exp OR death* OR 'survival'/exp OR survival) Screening and Data Extraction • Screening Process: o Two human reviewers will perform screening. A third reviewer will run the screening process using ChatGPT. o Blinding will be maintained across the entire screening process. • Data Extraction: o ChatGPT assists in extracting key data points from eligible studies, including study design, participant demographics, intervention details, and mortality outcomes. o Data extraction tables are reviewed and confirmed by human reviewers. Risk of Bias Assessment • The risk of bias will be evaluated using the Cochrane Collaboration's RoB 2.0 tool. • ChatGPT will initially perform the risk of bias assessment, providing detailed justifications for each decision, which will be reviewed by two independent experts for accuracy. Data Synthesis and Analysis • Statistical Analysis: o Meta-analysis will be conducted by an expert, using fixed-effects and random-effects models to calculate pooled hazard ratios (HRs) for all-cause and cardiovascular mortality. o Heterogeneity will be assessed using the I² statistic. • Sensitivity and Publication Bias Analysis: o Sensitivity analysis will be performed by omitting each study sequentially to assess the robustness of the results. o Publication bias will be evaluated using Egger's test and funnel plots. • AI-Execute Meta-Analysis o A custom GPT will be trained to generate the code to preform the meta-analysis in R. o The process will be run by a blinded reviewer and the results will be compared to the human executed meta-analysis 9. AI Methodology • AI Integration: o AI tools (ChatGPT) will be integrated into various stages of the systematic review process, including search strategy development, study screening, data extraction, risk of bias assessment, and meta-analysis. • Validation and Oversight: o Human reviewers will validate all AI-generated outputs to ensure accuracy and reliability. o Comparisons between AI-generated and human-conducted meta-analysis results will be performed to assess concordance. 10. Ethical Considerations • As this study involves the synthesis of published data, no ethical approval is required. 11. Dissemination Plans • The findings will be disseminated through publication in a peer-reviewed journal. 12. Funding • No specific funding was received for this study. 13. Conflicts of Interest • The authors declare no conflicts of interest related to this study. 14. Contact Information • Principal Investigator: Lefteris Teperikidis, Clinical Research Unit, Special Unit for Biomedical Research and Education (SUBRE), School of Medicine, Aristotle University of Thessaloniki, Thessaloniki, Greece. • Email: [email protected]

Efficacy of Biologic Agents and Small Molecules for Endoscopic Improvement and Mucosal Healing in Patients with Moderate-to-Severe Ulcerative Colitis: Systematic Review and Meta-Analysis

Background and Aim: The therapeutic landscape for ulcerative colitis (UC) is rapidly evolving, with an increasing number of biologic agents available. This systematic review and meta-analysis synthesized randomized controlled trials (RCTs) data on biologic therapies for achieving key endoscopic and histologic endpoints in moderate to severe UC. Methods: A systematic search of MEDLINE, EMBASE, Cochrane Library, Web of Science and grey literature was conducted through November 2024. Separate meta-analyses were performed for induction and maintenance. A random-effects model was used to estimate relative risks (RR), with 95% confidence intervals (CI), and confidence in estimates was evaluated with the GRADE approach (Grading of Recommendation Assessment, Development and Evaluation). Results: We included 40 RCTs (13 therapies, 14,369 patients). Thirty-two trials provided data in induction and twenty-eight in maintenance. During induction, all biologic therapies, except mirikizumab and filgotinib 100 mg, demonstrated superiority over placebo (RR 2.02, 95% CI: 1.76–2.31, I2 = 72%) for endoscopic improvement. Upadacitinib showed the highest efficacy (RR 5.53, 95% CI: 3.78–8.09). For mucosal healing, all interventions were superior to placebo (RR 2.95, 95% CI: 2.11–4.13, I2 = 61%), except filgotinib 100 mg. Risankizumab showed the highest efficacy (RR 10.25, 95% CI: 2.49–42.11). In maintenance, all therapies showed superiority over placebo for endoscopic improvement. For mucosal healing all therapies were superior to placebo, except risankizumab. Upadacitinib 30 mg showed the highest efficacy (RR 4.01, 95% CI: 1.81–8.87). Conclusions: Biologic and small-molecule therapies demonstrated substantial efficacy in achieving key endpoints. Standardized outcome definitions and further head-to-head RCTs are essential to strengthen confidence in our findings