The educational impact of assessment: a comparison of DOPS and MCQs

Aim: To evaluate the impact of two different assessment formats on the approaches to learning of final year veterinary students. The relationship between approach to learning and examination performance was also investigated. Method: An 18-item version of the Study Process Questionnaire (SPQ) was sent to 87 final year students. Each student responded to the questionnaire with regards to DOPS (Direct Observation of Procedural Skills) and a Multiple Choice Examination (MCQ). Semi-structured interviews were conducted with 16 of the respondents to gain a deeper insight into the students’ perception of assessment. Results: Students’ adopted a deeper approach to learning for DOPS and a more surface approach with MCQs. There was a positive correlation between an achieving approach to learning and examination performance. Analysis of the qualitative data revealed that deep, surface and achieving approaches were reported by the students and seven major influences on their approaches to learning were identified: motivation, purpose, consequence, acceptability, feedback, time pressure and the individual difference of the students. Conclusions: The format of DOPS has a positive influence on approaches to learning. There is a conflict for students between preparing for final examinations and preparing for clinical practice

Exploring medical student learning in the large group teaching environment: examining current practice to inform curricular development

Background Lectures continue to be an efficient and standardised way to deliver information to large groups of students. It has been well documented that students prefer interactive lectures, based on active learning principles, to didactic teaching in the large group setting. Despite this, it is often the case than many students do not engage with active learning tasks and attempts at interaction. By exploring student experiences, expectations and how they use lectures in their learning we will provide recommendations for faculty to support student learning both in the lecture theatre and during personal study time. Methods This research employed a hermeneutic phenomenological approach. Three focus groups, consisting of 19 students in total, were used to explore the experiences of second year medical students in large group teaching sessions. Using generic thematic data analysis, these accounts have been developed into a meaningful account of experience. Results This study found there to be a well-established learning culture amongst students and with it, expectations as to the format of teaching sessions. Furthermore, there were set perceptions about the student role within the learning environment which had many implications, including the way that innovative teaching methods were received. Student learning was perceived to take place outside the lecture theatre, with a large emphasis placed on creating resources that can be taken away to use in personal study time. Conclusions Presented here is a constructive review of reasons for student participation, interaction and engagement in large group teaching sessions. Based on this are recommendations constructed with the view to aid educators in engaging students within this setting. Short term, educators can implement strategies that monopolise on the established learning culture of students to encourage engagement with active learning strategies. Long term, it would be beneficial for educators to consider ways to shift the current student learning culture to one that embraces an active learning curriculum

Assessing fitness-to-practice of overseas-trained health practitioners by Australian registration & accreditation bodies

Assessment of fitness-to-practice of health professionals trained overseas and who wish to practice in Australia is undertaken by a range of organisations. These organisations conduct assessments using a range of methods. However there is very little published about how these organisations conduct their assessments. The purpose of the current paper is to investigate the methods of assessment used by these organisations and the issues associated with conducting these assessments

Medical students' and facilitators' experiences of an Early Professional Contact course: Active and motivated students, strained facilitators

Background: Today, medical students are introduced to patient contact, communication skills, and clinical examination in the preclinical years of the curriculum with the purpose of gaining clinical experience. These courses are often evaluated from the student perspective. Reports with an additional emphasis on the facilitator perspective are scarce. According to constructive alignment, an influential concept from research in higher education, the learning climate between students and teachers is also of great importance. In this paper, we approach the learning climate by studying both students' and facilitators' course experiences.\ud \ud In 2001, a new "Early Professional Contact" longitudinal strand through term 1–4, was introduced at the Sahlgrenska Academy, University of Gothenburg, Sweden. General practitioners and hospital specialists were facilitators.\ud \ud The aim of this study was to assess and analyse students' and clinical facilitators' experiences of the Early Professional Contact course and to illuminate facilitators' working conditions.\ud \ud Methods: Inspired by a Swedish adaptation of the Course Experience Questionnaire, an Early Professional Contact Questionnaire was constructed. In 2003, on the completion of the first longitudinal strand, a student and facilitator version was distributed to 86 students and 21 facilitators. In the analysis, both Chi-square and the Mann-Whitney tests were used.\ud \ud Results: Sixty students (70%) and 15 facilitators (71%) completed the questionnaire. Both students and facilitators were satisfied with the course. Students reported gaining [inspiration] for their future work as doctors along with increased confidence in meeting patients. They also reported increased motivation for biomedical studies. Differences in attitudes between facilitators and students were found. Facilitators experienced a greater workload, less reasonable demands and less support, than students.\ud \ud Conclusion: In this project, a new Early Professional Contact course was analysed from both student and facilitator perspectives. The students experienced the course as providing them with a valuable introduction to the physician's professional role in clinical practice. In contrast, course facilitators often experienced a heavy workload and lack of support, despite thorough preparatory education. A possible conflict between the clinical facilitator's task as educator and member of the workplace is suggested. More research is needed on how doctors combine their professional tasks with work as facilitators

"I couldn't do this with opposition from my colleagues": A qualitative study of physicians' experiences as clinical tutors

<p>Abstract</p> <p>Background</p> <p>Clinical contact in the early curriculum and workplace learning with active tutorship are important parts of modern medical education. In a previously published study, we found that medical students' tutors experienced a heavier workload, less reasonable demands and less encouragement, than students. The aim of this interview study was to further illuminate physicians' experiences as clinical tutors.</p> <p>Methods</p> <p>Twelve tutors in the Early Professional Contact course were interviewed. In the explorative interviews, they were asked to reflect upon their experiences of working as tutors in this course. Systematic text condensation was used as the analysis method.</p> <p>Results</p> <p>In the analysis, five main themes of physicians' experiences as clinical tutors in the medical education emerged: <it>(a) Pleasure and stimulation</it>. Informants appreciated tutorship and meeting both students and fellow tutors, <it>(b) Disappointment and stagnation</it>. Occasionally, tutors were frustrated and expressed negative feelings, <it>(c) Demands and duty</it>. Informants articulated an ambition to give students their best; a desire to provide better medical education but also a duty to meet demands of the course management, <it>(d) Impact of workplace relations</it>. Tutoring was made easier when the clinic's management provided active support and colleagues accepted students at the clinic, and <it>(e) Multitasking difficulties</it>. Combining several duties with those of a tutorship was often reported as difficult.</p> <p>Conclusions</p> <p>It is important that tutors' tasks are given adequate time, support and preparation. Accordingly, it appears highly important to avoid multitasking and too heavy a workload among tutors in order to facilitate tutoring. A crucial factor is acceptance and active organizational support from the clinic's management. This implies that tutoring by workplace learning in medical education should play an integrated and accepted role in the healthcare system.</p

Student evaluation of an OSCE in paediatrics at the University of the West Indies, Jamaica

BACKGROUND: The Faculty of Medical Sciences, University of the West Indies first implemented the Objective Structured Clinical Examination (OSCE) in the final MB Examination in Medicine and Therapeutics during the 2000–2001 academic year. Simultaneously, the Child Health Department initiated faculty and student training, and instituted the OSCE as an assessment instrument during the Child Health (Paediatric) clerkship in year 5. The study set out to explore student acceptance of the OSCE as part of an evaluation of the Child Health clerkship. METHODS: A self-administered questionnaire was completed by successive groups of students immediately after the OSCE at the end of each clerkship rotation. Main outcome measures were student perception of examination attributes, which included the quality of instructions and organisation, the quality of performance, authenticity and transparency of the process, and usefulness of the OSCE as an assessment instrument compared to other formats. RESULTS: There was overwhelming acceptance of the OSCE in Child Health with respect to the comprehensiveness (90%), transparency (87%), fairness (70%) and authenticity of the required tasks (58–78%). However, students felt that it was a strong anxiety-producing experience. And concerns were expressed regarding the ambiguity of some questions and inadequacy of time for expected tasks. CONCLUSION: Student feedback was invaluable in influencing faculty teaching, curriculum direction and appreciation of student opinion. Further psychometric evaluation will strengthen the development of the OSCE

Introducing a reward system in assessment in histology: A comment on the learning strategies it might engender

BACKGROUND: Assessment, as an inextricable component of the curriculum, is an important factor influencing student approaches to learning. If assessment is to drive learning, then it must assess the desired outcomes. In an effort to alleviate some of the anxiety associated with a traditional discipline-based second year of medical studies, a bonus system was introduced into the Histology assessment. Students obtaining a year mark of 70% were rewarded with full marks for some tests, resulting in many requiring only a few percentage points in the final examination to pass Histology. METHODS: In order to ascertain whether this bonus system might be impacting positively on student learning, thirty-two second year medical students (non-randomly selected, representing four academic groups based on their mid-year results) were interviewed in 1997 and, in 1999, the entire second year class completed a questionnaire (n = 189). Both groups were asked their opinions of the bonus system. RESULTS: Both groups overwhelming voted in favour of the bonus system, despite less than 45% of students failing to achieve it. Students commented that it relieved some of the stress of the year-end examinations, and was generally motivating with regard to their work commitment. CONCLUSIONS: Being satisfied with how and what we assess in Histology, we are of the opinion that this reward system may contribute to engendering appropriate learning approaches (i.e. for understanding) in students. As a result of its apparent positive influence on learning and attitudes towards learning, this bonus system will continue to operate until the traditional programme is phased out. It is hoped that other educators, believing that their assessment is a reflection of the intended outcomes, might recognise merit in rewarding students for consistent achievement

Assessment of examiner leniency and stringency ('hawk-dove effect') in the MRCP(UK) clinical examination (PACES) using multi-facet Rasch modelling

BACKGROUND: A potential problem of clinical examinations is known as the hawk-dove problem, some examiners being more stringent and requiring a higher performance than other examiners who are more lenient. Although the problem has been known qualitatively for at least a century, we know of no previous statistical estimation of the size of the effect in a large-scale, high-stakes examination. Here we use FACETS to carry out a multi-facet Rasch modelling of the paired judgements made by examiners in the clinical examination (PACES) of MRCP(UK), where identical candidates were assessed in identical situations, allowing calculation of examiner stringency. METHODS: Data were analysed from the first nine diets of PACES, which were taken between June 2001 and March 2004 by 10,145 candidates. Each candidate was assessed by two examiners on each of seven separate tasks. with the candidates assessed by a total of 1,259 examiners, resulting in a total of 142,030 marks. Examiner demographics were described in terms of age, sex, ethnicity, and total number of candidates examined. RESULTS: FACETS suggested that about 87% of main effect variance was due to candidate differences, 1% due to station differences, and 12% due to differences between examiners in leniency-stringency. Multiple regression suggested that greater examiner stringency was associated with greater examiner experience and being from an ethnic minority. Male and female examiners showed no overall difference in stringency. Examination scores were adjusted for examiner stringency and it was shown that for the present pass mark, the outcome for 95.9% of candidates would be unchanged using adjusted marks, whereas 2.6% of candidates would have passed, even though they had failed on the basis of raw marks, and 1.5% of candidates would have failed, despite passing on the basis of raw marks. CONCLUSION: Examiners do differ in their leniency or stringency, and the effect can be estimated using Rasch modelling. The reasons for differences are not clear, but there are some demographic correlates, and the effects appear to be reliable across time. Account can be taken of differences, either by adjusting marks or, perhaps more effectively and more justifiably, by pairing high and low stringency examiners, so that raw marks can be used in the determination of pass and fail

Using the Ages and Stages Questionnaire to teach medical students developmental assessment: a descriptive analysis

BACKGROUND: After a survey of medical graduates' skills found a lack of confidence in developmental assessment, a program was introduced with the broad aims of increasing medical student confidence and respect for the parents' role in childhood developmental assessment. Research has shown that parents' concerns are as accurate as quality screening tests in assessing development, so the program utilised the Ages and Stages Questionnaire, a parent completed, child development assessment tool. METHOD: To evaluate the program, an interpretative analysis was completed on the students' reports written during the program and a questionnaire was administered to the parents to gain their perception of the experience. As well, student confidence levels in assessing growth and development were measured at the end of the paediatric term. RESULTS: Although there was an increase in student confidence in developmental assessment at the end of the term, it was not statistically significant. However the findings indicated that students gained increased understanding of the process and enhanced recognition of the parental role, and the study suggested there was increased confidence in some students. Parents indicated that they thought they should be involved in the teaching of students. CONCLUSION: The ASQ was shown to have been useful in an education program at the level of advanced beginners in developmental assessment