An exploratory study into the effect of time-restricted internet access on face-validity, construct validity and reliability of postgraduate knowledge progress testing

BACKGROUND: Yearly formative knowledge testing (also known as progress testing) was shown to have a limited construct-validity and reliability in postgraduate medical education. One way to improve construct-validity and reliability is to improve the authenticity of a test. As easily accessible internet has become inseparably linked to daily clinical practice, we hypothesized that allowing internet access for a limited amount of time during the progress test would improve the perception of authenticity (face-validity) of the test, which would in turn improve the construct-validity and reliability of postgraduate progress testing. METHODS: Postgraduate trainees taking the yearly knowledge progress test were asked to participate in a study where they could access the internet for 30 minutes at the end of a traditional pen and paper test. Before and after the test they were asked to complete a short questionnaire regarding the face-validity of the test. RESULTS: Mean test scores increased significantly for all training years. Trainees indicated that the face-validity of the test improved with internet access and that they would like to continue to have internet access during future testing. Internet access did not improve the construct-validity or reliability of the test. CONCLUSION: Improving the face-validity of postgraduate progress testing, by adding the possibility to search the internet for a limited amount of time, positively influences test performance and face-validity. However, it did not change the reliability or the construct-validity of the test

Measuring cyber secure behavior of elementary and high school students in the Netherlands

School systems may pay attention to the fact that individuals and companies using smart devices are increasingly at risk of becoming victims of cybercrime. The literature on how effective students in developed countries such as the Netherlands are taught about cyber security skills during their school career is scarce. Although curriculum materials are available, scaling up computer science education is behind. Therefore, this study explores to what extent Dutch students develop cyber secure behavior at elementary and high school. A questionnaire was used for self-assessment of cyber security behavior. After the questionnaire was completed, two group interviews were conducted to improve the interpretation of the questionnaire results. The study findings revealed that the Dutch school curriculum hardly pays attention to this topic and that students acquire their online behavior mainly through experience, instructions on the internet, through parents, and through siblings. In addition, many students developed more reckless behavior over time. We recommend that cyber security education should start at elementary school as soon as children begin to use online equipment. A subject that deserves special attention is recognizing phishing emails and phishing websites. The learners should be convinced that risky behavior on the internet may turn against them and against the organization to which they belong

Change Management Support in Postgraduate Medical Education: A Change for the Better

Curriculum change is inevitably a part of postgraduate medical education (PGME) due to a necessity to rapidly adapt to changes in societal needs, educational philosophy and technological advances. Initiating, adopting as well as sustaining successful change can be very challenging especially in complex and time-constrained environments such as healthcare and PGME. Indeed, research has shown that educational changes do not always lead to the desired adjustments in practice. Surprisingly, implementation processes in healthcare and, more particularly, those in medical education are rarely supported by change management principles despite the scale and implications of curriculum reforms that justify guidance of such implementation processes. Insights from a change management perspective could help to smoothen the transition from theory to practice by guiding implementation processes and provide support in routinizing innovations in standard practice. A thorough description about change from an educational as well as a change management perspective is made, followed by the experiences with introducing change management principles into PGME. Lastly, the potential of change management principles for future changes in medical education, and their practical implications, is presented

Patient Feedback to Enhance Residents’ Learning: A Patient and a Resident Perspective

Patients are becoming more involved in healthcare, however, their involvement in postgraduate medical education (PGME) is often less prominent. We provide insight into patients’ and residents’ perspectives regarding possible topics for patient feedback, to increase its use and effectiveness in PGME. Semi-structured interviews with 20 purposefully sampled patients were done and 15 residents filled out a fully qualitative questionnaire. The sample size was not calculated as we aimed for data sufficiency. Content analysis was inspired by grounded theory. Topics mentioned by patients and residents were communication skills and communication of medical knowledge. While patients find organizational matters and personal aspects important topics, residents do not. Patients intend to provide feedback on task-, process-, and self-level, whereas residents do not wish to receive feedback on self-level. Topics mentioned by patients corresponded with various CanMEDS roles, that is, communicator, collaborator, professional, and leader. Feedback directed on task- and process-level would be of residents’ interest, including feedback on the physician-patient relationship and communication of medical knowledge. Patient feedback should not only focus on communication skills but also on other CanMEDS roles. To provide effective feedback and ensure that it remains at the level that enhances residents’ learning, patients should avoid giving feedback on self-level

Lessons learned spanning 17 years of experience with three consecutive nationwide competency based medical education training plans

IntroductionCurricula for postgraduate medical education have transformed since the introduction of competency based medical education (CBME). Postgraduate training plans offer broader training with different competencies and an outcome-based approach, in addition to the medical technical aspects of training. However, CBME also has its challenges. Over the past years, critical views have been shared on the potential drawbacks of CBME, such as assessment burden and conflicts with practicality in the workplace. Recent studies identified a need for a better understanding of how the evolving concept of CBME has been translated to curriculum design and implemented in the practice of postgraduate training. The aim of this study was to describe the development of CBME translations to curriculum design, based on three consecutive postgraduate training programs spanning 17 years.MethodWe performed a document analysis of three consecutive Dutch gynecology and obstetrics training plans that were implemented in 2005, 2013, and 2021. We used template analysis to identify changes over time.ResultsOver time, CBME-based curriculum design changed in several domains. Assessment changed from a model with a focus on summative decision to one with an emphasis on formative, low-stakes assessments aimed at supporting learning. The training plans evolved in parallel to evolving educational insights, e.g., by placing increasing emphasis on personal development. The curricula focused on a competency-based concept by introducing training modules and personalized authorization based on feedback rather than on a set duration of internships. There was increasing freedom in personalized training trajectories in the training plans, together with increasing trust towards the resident.ConclusionThe way CBME was translated into training plans has evolved in the course of 17 years of experience with CMBE-based education. The main areas of change were the structure of the training plans, which became increasingly open, the degree to which learning outcomes were mandatory or not, and the way these outcomes were assessed