Comparison between Long-Menu and Open-Ended Questions in computerized medical assessments. A randomized controlled trial

BACKGROUND: Long-menu questions (LMQs) are viewed as an alternative method for answering open-ended questions (OEQs) in computerized assessment. So far this question type and its influence on examination scores have not been studied sufficiently. However, the increasing use of computerized assessments will also lead to an increasing use of this question type. Using a summative online key feature (KF) examination we evaluated whether LMQs can be compared with OEQs in regard to the level of difficulty, performance and response times. We also evaluated the content for its suitability for LMQs. METHODS: We randomized 146 fourth year medical students into two groups. For the purpose of this study we created 7 peer-reviewed KF-cases with a total of 25 questions. All questions had the same content in both groups, but nine questions had a different answer type. Group A answered 9 questions with an LM type, group B with an OE type. In addition to the LM answer, group A could give an OE answer if the appropriate answer was not included in the list. RESULTS: The average number of correct answers for LMQs and OEQs showed no significant difference (p = 0.93). Among all 630 LM answers only one correct term (0.32%) was not included in the list of answers. The response time for LMQs did not significantly differ from that of OEQs (p = 0.65). CONCLUSION: LMQs and OEQs do not differ significantly. Compared to standard multiple-choice questions (MCQs), the response time for LMQs and OEQs is longer. This is probably due to the fact that they require active problem solving skills and more practice. LMQs correspond more suitable to Short answer questions (SAQ) then to OEQ and should only be used when the answers can be clearly phrased, using only a few, precise synonyms. LMQs can decrease cueing effects and significantly simplify the scoring in computerized assessment

Effect of running therapy on depression (EFFORT-D). Design of a randomised controlled trial in adult patients [ISRCTN 1894]

<p>Abstract</p> <p>Background</p> <p>The societal and personal burden of depressive illness is considerable. Despite the developments in treatment strategies, the effectiveness of both medication and psychotherapy is not ideal. Physical activity, including exercise, is a relatively cheap and non-harmful lifestyle intervention which lacks the side-effects of medication and does not require the introspective ability necessary for most psychotherapies. Several cohort studies and randomised controlled trials (RCTs) have been performed to establish the effect of physical activity on prevention and remission of depressive illness. However, recent meta-analysis's of all RCTs in this area showed conflicting results. The objective of the present article is to describe the design of a RCT examining the effect of exercise on depressive patients.</p> <p>Methods/Design</p> <p>The EFFect Of Running Therapy on Depression in adults (EFFORT-D) is a RCT, studying the effectiveness of exercise therapy (running therapy (RT) or Nordic walking (NW)) on depression in adults, in addition to usual care. The study population consists of patients with depressive disorder, Hamilton Rating Scale for Depression (HRSD) ≥ 14, recruited from specialised mental health care. The experimental group receives the exercise intervention besides treatment as usual, the control group receives treatment as usual. The intervention program is a group-based, 1 h session, two times a week for 6 months and of increasing intensity. The control group only performs low intensive non-aerobic exercises. Measurements are performed at inclusion and at 3,6 and 12 months.</p> <p>Primary outcome measure is reduction in depressive symptoms measured by the HRSD. Cardio-respiratory fitness is measured using a sub maximal cycling test, biometric information is gathered and blood samples are collected for metabolic parameters. Also, co-morbidity with pain, anxiety and personality traits is studied, as well as quality of life and cost-effectiveness.</p> <p>Discussion</p> <p>Exercise in depression can be used as a standalone or as an add-on intervention. In specialised mental health care, chronic forms of depression, co-morbid anxiety or physical complaints and treatment resistance are common. An add-on strategy therefore seems the best choice. This is the first high quality large trial into the effectiveness of exercise as an add-on treatment for depression in adult patients in specialised mental health care.</p> <p>Trial registration</p> <p>Netherlands Trial Register (NTR): <a href="http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=1894">NTR1894</a></p

Erratum to: Methods for evaluating medical tests and biomarkers

[This corrects the article DOI: 10.1186/s41512-016-0001-y.]

The effect of implementing undergraduate competency-based medical education on students' knowledge acquisition, clinical performance and perceived preparedness for practice:a comparative study

<p>Background: Little is known about the gains and losses associated with the implementation of undergraduate competency-based medical education. Therefore, we compared knowledge acquisition, clinical performance and perceived preparedness for practice of students from a competency-based active learning (CBAL) curriculum and a prior active learning (AL) curriculum.</p><p>Methods: We included two cohorts of both the AL curriculum (n = 453) and the CBAL curriculum (n = 372). Knowledge acquisition was determined by benchmarking each cohort on 24 interuniversity progress tests against parallel cohorts of two other medical schools. Differences in knowledge acquisition were determined comparing the number of times CBAL and AL cohorts scored significantly higher or lower on progress tests. Clinical performance was operationalized as students' mean clerkship grade. Perceived preparedness for practice was assessed using a survey.</p><p>Results: The CBAL cohorts demonstrated relatively lower knowledge acquisition than the AL cohorts during the first study years, but not at the end of their studies. We found no significant differences in clinical performance. Concerning perceived preparedness for practice we found no significant differences except that students from the CBAL curriculum felt better prepared for 'putting a patient problem in a broad context of political, sociological, cultural and economic factors' than students from the AL curriculum.</p><p>Conclusions: Our data do not support the assumption that competency-based education results in graduates who are better prepared for medical practice. More research is needed before we can draw generalizable conclusions on the potential of undergraduate competency-based medical education.</p>

Competency Assessment

Assessment is an essential feature of the competency-based educational model because only by means of evaluation can we verify achievement of specified learning outcomes. This is especially important in the context of health professions education, where the competencies of interest impact the well-being of patients. Therefore, just as with planning the instructional component of a curriculum, development of an assessment system must start with the specification of desired learning outcomes in the form of knowledge, skills, and attitudes expected of trainees or practitioners in order to provide safe and effective patient care. Issues to consider when judging the quality of evaluation methods include the reliability of data generated by the assessment, validity of decisions based on test results, educational impact on individuals undergoing evaluation and other stakeholders, and the feasibility of implementing the assessment system. In addition to these criteria and the particular competencies to be evaluated, the choice of testing methods from among numerous available techniques should consider multiple dimensions, such as appropriate level of assessment, stage of learner development, and, very importantly, overall purpose and context of the assessment. Ultimately, no one method can assess all aspects of professional competence, but familiarity with strengths and limitations of various modalities can guide the development of appropriate assessment systems. Strengths of simulation-based methods for evaluative purposes include the ability to assess actual performance of psychomotor skills and demonstration of nontechnical professional competencies in environments that safely and authentically mirror real practice settings. In addition, the programmability of simulations permits on-demand testing of rare but important clinical situations and consistent presentation of evaluation problems to multiple examinees; this reproducibility becomes especially important when high-stakes decisions are contingent upon such assessments