Academic Position

Director of the Alberta International Medical Graduate (AIMG) Progra

Academic Position in Medical Education Research

The Office of Post Graduate Medical Education (PGME), together with the Medical Education and Research Unit (MERU), invites applications for a full-time academic position in medical education research at the Assistant or Associate Professor level

Medical Education Breakthroughs: What Constitutes Milestones in Curriculum, Teaching and Assessment?

Medical Education Breakthroughs: What Constitutes Milestones in Curriculum, Teaching and Assessment

Predictive validity of the UK clinical aptitude test in the final years of medical school:a prospective cohort study

Peer reviewedPublisher PD

Large deviations of the maximal eigenvalue of random matrices

We present detailed computations of the 'at least finite' terms (three dominant orders) of the free energy in a one-cut matrix model with a hard edge a, in beta-ensembles, with any polynomial potential. beta is a positive number, so not restricted to the standard values beta = 1 (hermitian matrices), beta = 1/2 (symmetric matrices), beta = 2 (quaternionic self-dual matrices). This model allows to study the statistic of the maximum eigenvalue of random matrices. We compute the large deviation function to the left of the expected maximum. We specialize our results to the gaussian beta-ensembles and check them numerically. Our method is based on general results and procedures already developed in the literature to solve the Pastur equations (also called "loop equations"). It allows to compute the left tail of the analog of Tracy-Widom laws for any beta, including the constant term.Comment: 62 pages, 4 figures, pdflatex ; v2 bibliography corrected ; v3 typos corrected and preprint added ; v4 few more numbers adde

The academic backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors

Background: Selection of medical students in the UK is still largely based on prior academic achievement, although doubts have been expressed as to whether performance in earlier life is predictive of outcomes later in medical school or post-graduate education. This study analyses data from five longitudinal studies of UK medical students and doctors from the early 1970s until the early 2000s. Two of the studies used the AH5, a group test of general intelligence (that is, intellectual aptitude). Sex and ethnic differences were also analyzed in light of the changing demographics of medical students over the past decades. Methods: Data from five cohort studies were available: the Westminster Study (began clinical studies from 1975 to 1982), the 1980, 1985, and 1990 cohort studies (entered medical school in 1981, 1986, and 1991), and the University College London Medical School (UCLMS) Cohort Study (entered clinical studies in 2005 and 2006). Different studies had different outcome measures, but most had performance on basic medical sciences and clinical examinations at medical school, performance in Membership of the Royal Colleges of Physicians (MRCP(UK)) examinations, and being on the General Medical Council Specialist Register. Results: Correlation matrices and path analyses are presented. There were robust correlations across different years at medical school, and medical school performance also predicted MRCP(UK) performance and being on the GMC Specialist Register. A-levels correlated somewhat less with undergraduate and post-graduate performance, but there was restriction of range in entrants. General Certificate of Secondary Education (GCSE)/O-level results also predicted undergraduate and post-graduate outcomes, but less so than did A-level results, but there may be incremental validity for clinical and post-graduate performance. The AH5 had some significant correlations with outcome, but they were inconsistent. Sex and ethnicity also had predictive effects on measures of educational attainment, undergraduate, and post-graduate performance. Women performed better in assessments but were less likely to be on the Specialist Register. Non-white participants generally underperformed in undergraduate and post-graduate assessments, but were equally likely to be on the Specialist Register. There was a suggestion of smaller ethnicity effects in earlier studies. Conclusions: The existence of the Academic Backbone concept is strongly supported, with attainment at secondary school predicting performance in undergraduate and post-graduate medical assessments, and the effects spanning many years. The Academic Backbone is conceptualized in terms of the development of more sophisticated underlying structures of knowledge ('cognitive capital’ and 'medical capital’). The Academic Backbone provides strong support for using measures of educational attainment, particularly A-levels, in student selection

Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies

Background: Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities. Methods: Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation. Results: Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels. Conclusions: Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills

Using Item Analysis to Assess Objectively the Quality of the Calgary-Cambridge OSCE Checklist

Background:  The purpose of this study was to investigate the use of item analysis to assess objectively the quality of items on the Calgary-Cambridge Communications OSCE checklist.Methods:  A total of 150 first year medical students were provided with extensive teaching on the use of the Calgary-Cambridge Guidelines for interviewing patients and participated in a final year end 20 minute communication OSCE station.  Grouped into either the upper half (50%) or lower half (50%) communication skills performance groups, discrimination, difficulty and point biserial values were calculated for each checklist item.Results:  The mean score on the 33 item communication checklist was 24.09 (SD = 4.46) and the internal reliability coefficient was ? = 0.77. Although most of the items were found to have moderate (k = 12, 36%) or excellent (k = 10, 30%) discrimination values, there were 6 (18%) identified as ‘fair’ and 3 (9%) as ‘poor’. A post-examination review focused on item analysis findings resulted in an increase in checklist reliability (? = 0.80).Conclusions:  Item analysis has been used with MCQ exams extensively. In this study, it was also found to be an objective and practical approach to use in evaluating the quality of a standardized OSCE checklist

The Reliability and Validity of a Paediatric Script Concordance Test with Medical Students, Paediatric Residents and Experienced Paediatricians

Background: The Script Concordance (SC) approach was used as an alternative test format to measure the presence of knowledge organization reflective in one’s clinical reasoning skills (i.e., diagnostic, investigation and treatment knowledge).Methods:  The present study investigated the reliability and validity of a 40-item paediatric version of the SC test with three groups representing 53 medical students (novices), 42 paediatric residents (intermediates) and 11 paediatricians (experts).Results:  A comparison between scoring techniques based on experts’ ratings of the items showed internal reliability coefficients from .74 for the one-best answer up to .78 for alternative scoring techniques.  An ANOVA showed an increase in test performance from medical students through to expert paediatricians (F(2,103) = 84.05, p < .001), but did not differentiate between the postgraduate year 1 to 3 paediatric residents.  A large effect size (Cohen’s d) difference of 1.06 was found between medical students and residents total SC test scores.Conclusions:  These results support other findings indicating the SC test format can be used to differentiate between the clinical reasoning skills of novices, intermediates and experts in paediatrics.  An alternative scoring method that includes one best answer and partial marks was also supported for grading SC test items