Simpson's Paradox, Lord's Paradox, and Suppression Effects are the same phenomenon – the reversal paradox

This article discusses three statistical paradoxes that pervade epidemiological research: Simpson's paradox, Lord's paradox, and suppression. These paradoxes have important implications for the interpretation of evidence from observational studies. This article uses hypothetical scenarios to illustrate how the three paradoxes are different manifestations of one phenomenon – the reversal paradox – depending on whether the outcome and explanatory variables are categorical, continuous or a combination of both; this renders the issues and remedies for any one to be similar for all three. Although the three statistical paradoxes occur in different types of variables, they share the same characteristic: the association between two variables can be reversed, diminished, or enhanced when another variable is statistically controlled for. Understanding the concepts and theory behind these paradoxes provides insights into some controversial or contradictory research findings. These paradoxes show that prior knowledge and underlying causal theory play an important role in the statistical modelling of epidemiological data, where incorrect use of statistical models might produce consistent, replicable, yet erroneous results

Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies

Background: Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities. Methods: Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation. Results: Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels. Conclusions: Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills

A latent trait look at pretest-posttest validation of criterion-referenced test items

Since Cox and Vargas (1966) introduced their pretest-posttest validity index for criterion-referenced test items, a great number of additions and modifications have followed. All are based on the idea of gain scoring; that is, they are computed from the differences between proportions of pretest and posttest item responses. Although the method is simple and generally considered as the prototype of criterion-referenced item analysis, it has many and serious disadvantages. Some of these go back to the fact that it leads to indices based on a dual test administration- and population-dependent item p values. Others have to do with the global information about the discriminating power that these indices provide, the implicit weighting they suppose, and the meaningless maximization of posttest scores they lead to. Analyzing the pretest-posttest method from a latent trait point of view, it is proposed to replace indices like Cox and Vargas’ Dpp by an evaluation of the item information function for the mastery score. An empirical study was conducted to compare the differences in item selection between both methods

ERCC1 expression and RAD51B activity correlate with cell cycle response to platinum drug treatment not DNA repair

Background: The H69CIS200 and H69OX400 cell lines are novel models of low-level platinum-drug resistance. Resistance was not associated with increased cellular glutathione or decreased accumulation of platinum, rather the resistant cell lines have a cell cycle alteration allowing them to rapidly proliferate post drug treatment. Results: A decrease in ERCC1 protein expression and an increase in RAD51B foci activity was observed in association with the platinum induced cell cycle arrest but these changes did not correlate with resistance or altered DNA repair capacity. The H69 cells and resistant cell lines have a p53 mutation and consequently decrease expression of p21 in response to platinum drug treatment, promoting progression of the cell cycle instead of increasing p21 to maintain the arrest. Conclusion: Decreased ERCC1 protein and increased RAD51B foci may in part be mediating the maintenance of the cell cycle arrest in the sensitive cells. Resistance in the H69CIS200 and H69OX400 cells may therefore involve the regulation of ERCC1 and RAD51B independent of their roles in DNA repair. The novel mechanism of platinum resistance in the H69CIS200 and H69OX400 cells demonstrates the multifactorial nature of platinum resistance which can occur independently of alterations in DNA repair capacity and changes in ERCC1

Augmented visual feedback of movement performance to enhance walking recovery after stroke : study protocol for a pilot randomised controlled trial

Increasing evidence suggests that use of augmented visual feedback could be a useful approach to stroke rehabilitation. In current clinical practice, visual feedback of movement performance is often limited to the use of mirrors or video. However, neither approach is optimal since cognitive and self-image issues can distract or distress patients and their movement can be obscured by clothing or limited viewpoints. Three-dimensional motion capture has the potential to provide accurate kinematic data required for objective assessment and feedback in the clinical environment. However, such data are currently presented in numerical or graphical format, which is often impractical in a clinical setting. Our hypothesis is that presenting this kinematic data using bespoke visualisation software, which is tailored for gait rehabilitation after stroke, will provide a means whereby feedback of movement performance can be communicated in a more meaningful way to patients. This will result in increased patient understanding of their rehabilitation and will enable progress to be tracked in a more accessible way. The hypothesis will be assessed using an exploratory (phase II) randomised controlled trial. Stroke survivors eligible for this trial will be in the subacute stage of stroke and have impaired walking ability (Functional Ambulation Classification of 1 or more). Participants (n = 45) will be randomised into three groups to compare the use of the visualisation software during overground physical therapy gait training against an intensity-matched and attention-matched placebo group and a usual care control group. The primary outcome measure will be walking speed. Secondary measures will be Functional Ambulation Category, Timed Up and Go, Rivermead Visual Gait Assessment, Stroke Impact Scale-16 and spatiotemporal parameters associated with walking. Additional qualitative measures will be used to assess the participant's experience of the visual feedback provided in the study. Results from the trial will explore whether the early provision of visual feedback of biomechanical movement performance during gait rehabilitation demonstrates improved mobility outcomes after stroke and increased patient understanding of their rehabilitation

Early rheumatoid arthritis is characterized by a distinct and transient synovial fluid cytokine profile of T cell and stromal cell origin

Pathological processes involved in the initiation of rheumatoid synovitis remain unclear. We undertook the present study to identify immune and stromal processes that are present soon after the clinical onset of rheumatoid arthritis ( RA) by assessing a panel of T cell, macrophage, and stromal cell related cytokines and chemokines in the synovial fluid of patients with early synovitis. Synovial fluid was aspirated from inflamed joints of patients with inflammatory arthritis of duration 3 months or less, whose outcomes were subsequently determined by follow up. For comparison, synovial fluid was aspirated from patients with acute crystal arthritis, established RA and osteoarthritis. Rheumatoid factor activity was blocked in the synovial fluid samples, and a panel of 23 cytokines and chemokines measured using a multiplex based system. Patients with early inflammatory arthritis who subsequently developed RA had a distinct but transient synovial fluid cytokine profile. The levels of a range of T cell, macrophage and stromal cell related cytokines ( e. g. IL-2, IL-4, IL-13, IL-17, IL-15, basic fibroblast growth factor and epidermal growth factor) were significantly elevated in these patients within 3 months after symptom onset, as compared with early arthritis patients who did not develop RA. In addition, this profile was no longer present in established RA. In contrast, patients with non-rheumatoid persistent synovitis exhibited elevated levels of interferon-gamma at initiation. Early synovitis destined to develop into RA is thus characterized by a distinct and transient synovial fluid cytokine profile. The cytokines present in the early rheumatoid lesion suggest that this response is likely to influence the microenvironment required for persistent RA

Decoding machine learning benchmarks

Despite the availability of benchmark machine learning (ML) repositories (e.g., UCI, OpenML), there is no standard evaluation strategy yet capable of pointing out which is the best set of datasets to serve as gold standard to test different ML algorithms. In recent studies, Item Response Theory (IRT) has emerged as a new approach to elucidate what should be a good ML benchmark. This work applied IRT to explore the well-known OpenML-CC18 benchmark to identify how suitable it is on the evaluation of classifiers. Several classifiers ranging from classical to ensembles ones were evaluated using IRT models, which could simultaneously estimate dataset difficulty and classifiers' ability. The Glicko-2 rating system was applied on the top of IRT to summarize the innate ability and aptitude of classifiers. It was observed that not all datasets from OpenML-CC18 are really useful to evaluate classifiers. Most datasets evaluated in this work (84%) contain easy instances in general (e.g., around 10% of difficult instances only). Also, 80% of the instances in half of this benchmark are very discriminating ones, which can be of great use for pairwise algorithm comparison, but not useful to push classifiers abilities. This paper presents this new evaluation methodology based on IRT as well as the tool decodIRT, developed to guide IRT estimation over ML benchmarks.Comment: Paper published at the BRACIS 2020 conference, 15 pages, 4 figure

Radiogenomic analysis of primary breast cancer reveals [18F]-fluorodeoxglucose dynamic flux-constants are positively associated with immune pathways and outperform static uptake measures in associating with glucose metabolism

Background: PET imaging of 18F-fluorodeoxygucose (FDG) is used widely for tumour staging and assessment of treatment response, but the biology associated with FDG uptake is still not fully elucidated. We therefore carried out gene set enrichment analyses (GSEA) of RNA sequencing data to find KEGG pathways associated with FDG uptake in primary breast cancers. Methods: Pre-treatment data were analysed from a window-of-opportunity study in which 30 patients underwent static and dynamic FDG-PET and tumour biopsy. Kinetic models were fitted to dynamic images, and GSEA was performed for enrichment scores reflecting Pearson and Spearman coefficients of correlations between gene expression and imaging. Results: A total of 38 pathways were associated with kinetic model flux-constants or static measures of FDG uptake, all positively. The associated pathways included glycolysis/gluconeogenesis (‘GLYC-GLUC’) which mediates FDG uptake and was associated with model flux-constants but not with static uptake measures, and 28 pathways related to immune-response or inflammation. More pathways, 32, were associated with the flux-constant K of the simple Patlak model than with any other imaging index. Numbers of pathways categorised as being associated with individual micro-parameters of the kinetic models were substantially fewer than numbers associated with flux-constants, and lay around levels expected by chance. Conclusions: In pre-treatment images GLYC-GLUC was associated with FDG kinetic flux-constants including Patlak K, but not with static uptake measures. Immune-related pathways were associated with flux-constants and static uptake. Patlak K was associated with more pathways than were the flux-constants of more complex kinetic models. On the basis of these results Patlak analysis of dynamic FDG-PET scans is advantageous, compared to other kinetic analyses or static imaging, in studies seeking to infer tumour-to-tumour differences in biology from differences in imaging. Trial registration NCT01266486, December 24th 2010