Adding authenticity to controlled conditions assessment: introduction of an online, open book, essay based exam

© 2018, The Author(s). In the practice of designing examinations, where the emphasis is normally on the assessment of knowledge and understanding of content covered, the authenticity of that assessment model related to skills required for employability, as well as our growing access to instant information, is often missed. Some traditional methods of this form of assessment, particularly the paper based essay exam, have expectations of writing quality and structure as well as of knowledge and understanding, but without providing the tools with which to appropriately meet those expectations. To address this, a computer based, unseen, essay format exam was introduced to a final year module for applied science students, where access was permitted to specific online journal resources. Students wrote their submissions under controlled conditions in Microsoft Word and submitted it through the Virtual Learning Environment (VLE), Blackboard for grading. In addition to an improvement in average mark across the cohort compared to the previous year of paper based essay examinations on the same module, student feedback highlighted an improvement in their ability to organise and arrange their knowledge, and the usefulness of using research material to better evidence their own knowledge. Feedback, being completed online and returned to the students also provided greater understanding of grades achieved and of how to improve for future examinations, a unusual process for exams where graded papers are not normally returned. Though upskilling of staff is likely required for more widespread use, and repeat cohorts will no doubt demonstrate further pros and cons of this form of assessment, the model presented here shows promise for assessment of student communication of depth and breadth of knowledge, and of demonstrating skills more relevant to future employment

Evaluation of prognostic risk models for postoperative pulmonary complications in adult patients undergoing major abdominal surgery: a systematic review and international external validation cohort study

Background Stratifying risk of postoperative pulmonary complications after major abdominal surgery allows clinicians to modify risk through targeted interventions and enhanced monitoring. In this study, we aimed to identify and validate prognostic models against a new consensus definition of postoperative pulmonary complications. Methods We did a systematic review and international external validation cohort study. The systematic review was done in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We searched MEDLINE and Embase on March 1, 2020, for articles published in English that reported on risk prediction models for postoperative pulmonary complications following abdominal surgery. External validation of existing models was done within a prospective international cohort study of adult patients (≥18 years) undergoing major abdominal surgery. Data were collected between Jan 1, 2019, and April 30, 2019, in the UK, Ireland, and Australia. Discriminative ability and prognostic accuracy summary statistics were compared between models for the 30-day postoperative pulmonary complication rate as defined by the Standardised Endpoints in Perioperative Medicine Core Outcome Measures in Perioperative and Anaesthetic Care (StEP-COMPAC). Model performance was compared using the area under the receiver operating characteristic curve (AUROCC). Findings In total, we identified 2903 records from our literature search; of which, 2514 (86·6%) unique records were screened, 121 (4·8%) of 2514 full texts were assessed for eligibility, and 29 unique prognostic models were identified. Nine (31·0%) of 29 models had score development reported only, 19 (65·5%) had undergone internal validation, and only four (13·8%) had been externally validated. Data to validate six eligible models were collected in the international external validation cohort study. Data from 11 591 patients were available, with an overall postoperative pulmonary complication rate of 7·8% (n=903). None of the six models showed good discrimination (defined as AUROCC ≥0·70) for identifying postoperative pulmonary complications, with the Assess Respiratory Risk in Surgical Patients in Catalonia score showing the best discrimination (AUROCC 0·700 [95% CI 0·683–0·717]). Interpretation In the pre-COVID-19 pandemic data, variability in the risk of pulmonary complications (StEP-COMPAC definition) following major abdominal surgery was poorly described by existing prognostication tools. To improve surgical safety during the COVID-19 pandemic recovery and beyond, novel risk stratification tools are required. Funding British Journal of Surgery Society

Comparison of institutional innovation: Two universities' nurturing of computer-based examinations

University students are rarely permitted to use their own personal computers in ex-aminations. Yet this bring-your-own-device strategy appears essential for an eco-nomically sustainable transition to high stakes assessment using these ubiquitous professional tools. The focus is away from internet-based online testing, and on lo-cal use of personal computers isolated from networking infrastructure (to prevent collusion), under the watchful eye of invigilators. Our respective institutions are lead-ing this transition in the United Kingdom and Australia, so we report on the gradual nature of the transition and the technologies involved. Further, we provide a compar-ison of university student perceptions about essay-style writing by hand or keyboard, and willingness to adopt computer-based examinations. Generally institutions and students prefer a graduated transition through paper-replacement examinations with free choice of writing implement, before moving to compulsory computer use and questions incorporating multimedia and software use. The paper concludes with recommendations for consideration by other universities considering adopting com-puters in high stakes assessments

Computers in the university exam: Reflections from Scotland and Tasmania

No description availabl

Writing e-Exams in Pre-University College

Part 7: Self-assessment, e-Assessment and e-ExaminationsInternational audienceThis study examined students’ expressed strategies, habits and preferences with respect to responding to supervised text-based assessments. Two trials of a computerised examination system took place in an Australian pre-university college in 2016 and 2017. Students in several classes studying geography and globalisation completed a sequence of practice and assessed work. Data were collected using pre- and post-surveys about their preferred writing styles, habits and strategies in light of their choice to type or handwrite essay and short answer examinations. Comparisons were made between those that elected to handwrite and those who chose to type the examination, with several areas being significant. The performance (grades), production (word count) of the typists and hand-writers were also correlated and compared