Invariance testing for machine learning models

Machine learning models are expected to produce consistent results when the input data objects are slightly modified whilst the ground truth remains unchanged. Such characteristics are also known as invariance qualities. However, previous studies assessing ML model invariance have primarily relied on one or a few accuracy metrics, which provide limited information on how models behave under different transformations. This thesis addresses this gap by proposing a novel, systematic invariance testing framework that moves beyond aggregated accuracy scores and offers more detailed visual information to enable more complex and explainable analysis of model performance. Our framework introduces a generic and extendable methodology for evaluating invariance across various attributes, including simple transformations (e.g., object rotation) as well as complex environmental variations (e.g., background changes). It is designed to be fully automated, enabling extensive and reproducible testing while ensuring consistent and explainable assessments of models' invariance qualities. A key challenge in invariance testing is ordering and structuring test data, especially when testing an entire input space is infeasible. To address this, we develop a novel approach to sampling large information spaces, leveraging context-sensitive training data to construct context-free models. This method ensures that our framework can effectively assess invariance properties without requiring exhaustive test cases. By providing visualized patterns of model behavior, our approach enhances diagnostic capabilities, making it a valuable tool for both researchers and practitioners. Our findings demonstrate that the evaluation of ML models' invariance qualities based on the visualized patterns can be: more informative, automatic, reliable and consistent. This thesis contributes to the broader effort of improving ML model evaluation, ensuring that claims of invariance are substantiated with comprehensive and reproducible evidence

CuriousLLM: Elevating Multi-Document QA with Reasoning-Infused Knowledge Graph Prompting

In the field of Question Answering (QA), unifying large language models (LLMs) with external databases has shown great success. However, these methods often fall short in providing the advanced reasoning needed for complex QA tasks. To address these issues, we improve over a novel approach called Knowledge Graph Prompting (KGP), which combines knowledge graphs with a LLM-based agent to improve reasoning and search accuracy. Nevertheless, the original KGP framework necessitates costly fine-tuning with large datasets yet still suffers from LLM hallucination. Therefore, we propose a reasoning-infused LLM agent to enhance this framework. This agent mimics human curiosity to ask follow-up questions to more efficiently navigate the search. This simple modification significantly boosts the LLM performance in QA tasks without the high costs and latency associated with the initial KGP framework. Our ultimate goal is to further develop this approach, leading to more accurate, faster, and cost-effective solutions in the QA domain

Image similarity using an ensemble of context-sensitive models

Image similarity has been extensively studied in computer vision. In recent years, machine-learned models have shown their ability to encode more semantics than traditional multivariate metrics. However, in labelling semantic similarity, assigning a numerical score to a pair of images is impractical, making the improvement and comparisons on the task difficult. In this work, we present a more intuitive approach to build and compare image similarity models based on labelled data in the form of A:R vs B:R, i.e., determining if an image A is closer to a reference image R than another image B. We address the challenges of sparse sampling in the image space (R, A, B) and biases in the models trained with context-based data by using an ensemble model. Our testing results show that the ensemble model constructed performs ∼5% better than the best individual context-sensitive models. They also performed better than the models that were directly fine-tuned using mixed imagery data as well as existing deep embeddings, e.g., CLIP [30] and DINO [3]. This work demonstrates that context-based labelling and model training can be effective when an appropriate ensemble approach is used to alleviate the limitation due to sparse sampling

Power Control in Two-Tier Femtocell Networks

In a two tier cellular network -- comprised of a central macrocell underlaid with shorter range femtocell hotspots -- cross-tier interference limits overall capacity with universal frequency reuse. To quantify near-far effects with universal frequency reuse, this paper derives a fundamental relation providing the largest feasible cellular Signal-to-Interference-Plus-Noise Ratio (SINR), given any set of feasible femtocell SINRs. We provide a link budget analysis which enables simple and accurate performance insights in a two-tier network. A distributed utility-based SINR adaptation at femtocells is proposed in order to alleviate cross-tier interference at the macrocell from cochannel femtocells. The Foschini-Miljanic (FM) algorithm is a special case of the adaptation. Each femtocell maximizes their individual utility consisting of a SINR based reward less an incurred cost (interference to the macrocell). Numerical results show greater than 30% improvement in mean femtocell SINRs relative to FM. In the event that cross-tier interference prevents a cellular user from obtaining its SINR target, an algorithm is proposed that reduces transmission powers of the strongest femtocell interferers. The algorithm ensures that a cellular user achieves its SINR target even with 100 femtocells/cell-site, and requires a worst case SINR reduction of only 16% at femtocells. These results motivate design of power control schemes requiring minimal network overhead in two-tier networks with shared spectrum.Comment: 29 pages, 10 figures, Revised and resubmitted to the IEEE Transactions on Wireless Communication

Progress in prevention and treatment of Chikungunya fever

Chikungunya fever (CHIKF) is an infectious disease caused by Chikungunya virus (CHIKV) infection. The CHIKV is mainly transmitted through the bites of Aedes mosquitoes. The acute symptoms caused by the CHIKV are mainly characterized by fever, rash and severe joint pain. Two vaccines (IXCHIQ and VIMKUNYA) have now been approved for the prevention of CHIKF in key populations, but they have not yet been widely used. At present, there are no specific antiviral drugs approved for marketing against the CHIKV, and clinical treatment mainly focuses on symptomatic supportive therapy. Based on a review of the epidemiology, transmission risks, and prevention and control strategies of the CHIKV, this article focuses on describing the latest progress in the use of Chikungunya vaccines and treatment management

Denosumab combined with microwave ablation excisional scraping for giant cell tumor of the thoracic spine: a case report and literature review

Denosumab has recently become an important part of the treatment regime for spinal giant cell tumors of the bone (GCTB). Its use has significantly reduced the risk of surgery and postoperative complications in patients with spinal GCTB. However, the use of denosumab has not yet been optimized to reap the maximum benefits. Here, we have reported the case of a patient who was treated with denosumab in combination with excision and scraping for GCTB of the T10 vertebrae, which achieved good tumor control and no recurrence at the 2-year postoperative follow-up. We have also reviewed the case in the light of relevant literature as well as presented our ideas and recommendations for the optimal use of denosumab

The RCSB Protein Data Bank: a redesigned query system and relational database based on the mmCIF schema

The Protein Data Bank (PDB) is the central worldwide repository for three-dimensional (3D) structure data of biological macromolecules. The Research Collaboratory for Structural Bioinformatics (RCSB) has completely redesigned its resource for the distribution and query of 3D structure data. The re-engineered site is currently in public beta test at http://pdbbeta.rcsb.org. The new site expands the functionality of the existing site by providing structure data in greater detail and uniformity, improved query and enhanced analysis tools. A new key feature is the integration and searchability of data from over 20 other sources covering genomic, proteomic and disease relationships. The current capabilities of the re-engineered site, which will become the RCSB production site at http://www.pdb.org in late 2005, are described