Usability Testing a Potentially Inappropriate Medication Dashboard: A Core Component of the Dashboard Development Process

Abstract Background With the increased usage of dashboard reporting systems to monitor and track patient panels by clinical users, developers must ensure that the information displays they produce are accurate and intuitive. When evaluating usability of a clinical dashboard among potential end users, developers oftentimes rely on methods such as questionnaires as opposed to other, more time-intensive strategies that incorporate direct observation. Objectives Prior to release of the potentially inappropriate medication (PIM) clinical dashboard, designed to facilitate completion of a quality improvement project by clinician scholars enrolled in the Veterans Affairs (VA) workforce development Geriatric Scholars Program (GSP), we evaluated the usability of the system. This article describes the process of usability testing a dashboard reporting system with clinicians using direct observation and think-aloud moderating techniques. Methods We developed a structured interview protocol that combines virtual observation, think-aloud moderating techniques, and retrospective questioning of the overall user experience, including use of the System Usability Scale (SUS). Thematic analysis was used to analyze field notes from the interviews of three GSP alumni. Results Our structured approach to usability testing identified specific functional problems with the dashboard reporting system that were missed by results from the SUS. Usability testing lead to overall improvements in the intuitive use of the system, increased data transparency, and clarification of the dashboard's purpose. Conclusion Reliance solely on questionnaires and surveys at the end stages of dashboard development can mask potential functional problems that will impede proper usage and lead to misinterpretation of results. A structured approach to usability testing in the developmental phase is an important tool for developers of clinician friendly systems for displaying easily digested information and tracking outcomes for the purpose of quality improvement.</jats:p

Multiple clinical episodes of Plasmodium falciparum malaria in a low transmission intensity setting: exposure versus immunity

BACKGROUND: Epidemiological studies indicate that some children experience many more episodes of clinical malaria than their age mates in a given location. Whether this is as a result of the micro-heterogeneity of malaria transmission with some children effectively getting more exposure to infectious mosquitoes than others, or reflects a failure in the acquisition of immunity needs to be elucidated. Here, we investigated the determinants of increased susceptibility to clinical malaria by comparing the intensity of exposure to Plasmodium falciparum and the acquisition of immunity in children at the extreme ends of the over-dispersed distribution of the incidence of clinical malaria. METHODS: The study was nested within a larger cohort in an area where the intensity of malaria transmission was low. We identified children who over a five-year period experienced 5 to 16 clinical malaria episodes (children at the tail-end of the over-dispersed distribution, n = 35), remained malaria-free (n = 12) or had a single episode (n = 26). We quantified antibodies against seven Plasmodium falciparum merozoite antigens in plasma obtained at six cross-sectional surveys spanning these five years. We analyzed the antibody responses to identify temporal dynamics that associate with disease susceptibility. RESULTS: Children experiencing multiple episodes of malaria were more likely to be parasite positive by microscopy at cross-sectional surveys (X(2) test for trend 14.72 P = 0.001) and had a significantly higher malaria exposure index, than those in the malaria-free or single episode groups (Kruskal-Wallis test P = 0.009). In contrast, the five-year temporal dynamics of anti-merozoite antibodies were similar in the three groups. Importantly in all groups, antibody levels were below the threshold concentrations previously observed to be correlated with protective immunity. CONCLUSIONS: We conclude that in the context of a low malaria transmission setting, susceptibility to clinical malaria is not accounted for by anti-merozoite antibodies but appears to be a consequence of increased parasite exposure. We hypothesize that intensive exposure is a prerequisite for protective antibody concentrations, while little to modest exposure may manifest as multiple clinical infections with low levels of antibodies. These findings have implications for interventions that effectively lower malaria transmission intensity. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12916-015-0354-z) contains supplementary material, which is available to authorized users