Critical appraisal skills of family physicians in Ontario, Canada

BACKGROUND: Our primary objective in this study was to measure family physicians' knowledge of the key elements that go into assessing the validity and interpreting the results in three different types of studies: i) a randomized controlled trial (RCT); ii) a study evaluating a diagnostic test; and iii) a systematic review (SR). Our secondary objectives were to determine the relationship between the above skills and age, gender, and type of practice. METHODS: We obtained a random sample of 1000 family physicians in Ontario from the College of Family Physicians of Canada database. These physicians were sent a questionnaire in the mail with follow-up mailings to non-responders at 3 and 8 weeks. The questionnaire was designed to measure knowledge and understanding of the basic concepts of critical appraisal skills. Based on the responses to the questions an Evidence Based Medicine (EBM) Knowledge Score was determined for each physician. RESULTS: A response rate of 30.2% was achieved. The respondents were younger and more likely to be recent graduates than the population of Ontario Family Physicians as a whole. This was an expected outcome. Just over 50% of respondents were able to answer questions concerning the critical appraisal of methods and the interpretation of results of research articles satisfactorily. The average score on the 12-point EBM Knowledge Scale was 6.4. The younger physicians scored higher than the older physicians, and academic physicians scored higher than community-based physicians. Scores of male and female physicians did not differ. CONCLUSIONS: We have shown that in a population of physicians which is younger than the general population of physicians, about 50% have reasonable knowledge regarding the critical appraisal of the methods and the interpretation of results of a research article. In general, younger physicians were more knowledgeable than were older physicians. EBM principles were felt to be important to the practice of medicine by 95% of respondents

Assessing competency in Evidence Based Practice: strengths and limitations of current tools in practice

<p>Abstract</p> <p>Background</p> <p>Evidence Based Practice (EBP) involves making clinical decisions informed by the most relevant and valid evidence available. Competence can broadly be defined as a concept that incorporates a variety of domains including knowledge, skills and attitudes. Adopting an evidence-based approach to practice requires differing competencies across various domains including literature searching, critical appraisal and communication. This paper examines the current tools available to assess EBP competence and compares their applicability to existing assessment techniques used in medicine, nursing and health sciences.</p> <p>Discussion</p> <p>Only two validated assessment tools have been developed to specifically assess all aspects of EBP competence. Of the two tools (<it>Berlin </it>and <it>Fresno </it>tools), only the <it>Fresno </it>tool comprehensively assesses EBP competency across all relevant domains. However, both tools focus on assessing EBP competency in medical students; therefore neither can be used for assessing EBP competency across different health disciplines. The Objective Structured Clinical Exam (OSCE) has been demonstrated as a reliable and versatile tool to assess clinical competencies, practical and communication skills. The OSCE has scope as an alternate method for assessing EBP competency, since it combines assessment of cognitive skills including knowledge, reasoning and communication. However, further research is needed to develop the OSCE as a viable method for assessing EBP competency.</p> <p>Summary</p> <p>Demonstrating EBP competence is a complex task – therefore no single assessment method can adequately provide all of the necessary data to assess complete EBP competence. There is a need for further research to explore how EBP competence is best assessed; be it in written formats, such as the <it>Fresno </it>tool, or another format, such as the OSCE. Future tools must also incorporate measures of assessing how EBP competence affects clinician behaviour and attitudes as well as clinical outcomes in real-time situations. This research should also be conducted across a variety of health disciplines to best inform practice.</p