Methodological reflections on the evaluation of the implementation and adoption of national electronic health record systems

Copyright @ 2012, International Journal of Integrated Care (IJIC). This work is licensed under a (http://creativecommons.org/licenses/by/3.0) Creative Commons Attribution 3.0 Unported License.Introduction/purpose of presentation: Far-reaching policy commitments to information technology-centered transformations of healthcare systems have now been made in many countries. There is as yet little empirical evidence to justify such decisions, hence the need for rigorous independent evaluation of current implementation efforts. Such evaluations however pose a number of important challenges. This presentation has been designed as a part of a Panel based on our experience of evaluating the National Health Service’s (NHS) implementation of electronic health records (EHR) systems in hospitals throughout England. We discuss the methodological challenges encountered in planning and undertaking an evaluation of a program of this scale and reflect on why and how we adapted our evaluation approach—both conceptually and methodologically—in response to these challenges. Study design/population studied: Critical reflections on a multi-disciplinary and multi-facet independent evaluation of a national program to implement electronic health record systems into 12 ‘early wave’ NHS hospitals in England. Findings: Our initial plan was to employ a mixed methods longitudinal ‘before-during-after’ study design. We however found this unsustainable in the light of fluxes in policy, contractual issues and over-optimistic schedules for EHR deployments. More importantly, this research design failed adequately to address the core of multi-faceted evolving EHRs as understood by key stakeholders and as worked out in their distinct work settings. Thus conventional outcomes-centric evaluations may not easily scale-up when evaluating transformational programs and may indeed prove misleading. New assumptions concerning the implementation process of EHR need to be developed that recognize the constantly changing milieu of policy, product, projects and professions that are inherent to such national implementations. The approaches we subsequently developed substitute the positivist view that EHR initiatives are self-evident and self-contained interventions, which are amenable to traditional quantitative evaluations, to one that focuses on how they are understood by various stakeholders and made to work in specific contexts. These assumptions recast the role of evaluation towards an approach that explores and interprets processes of socio-technical change that surround EHR implementation and adoption as seen by multiple stakeholders. Conclusions and policy implications: There is likely to be an increase in politically-driven national programs of reform of healthcare based on information and communication technologies. Programs on such a scale are inherently complex with extended temporalities and extensive and dynamic sets of stakeholders. They are, in short, different and pose new evaluation challenges that previously formulated evaluation methods for health information systems cannot easily address. This calls for methodological innovation amongst research teams and their supporting bodies. We argue that evaluation of such system-wide transformation programs are likely to demand both breadth and depth of experience within a multidisciplinary research team, constant questioning of what is and what can be evaluated and how, and a particular way of working that emphasizes continuous dialogue and reflexivity. Making this transition is essential to enable evaluations that can usefully inform policy-making. Health policy experts urgently need to reassess the evaluation strategies they employ as they come to address national policies for system-wide transformation based on new electronic health infrastructures

Implementation and adoption of nationwide electronic health records in secondary care in England: Final qualitative results from prospective national evaluation in "early adopter" hospitals

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.Objectives: To evaluate the implementation and adoption of the NHS detailed care records service in “early adopter” hospitals in England. Design: Theoretically informed, longitudinal qualitative evaluation based on case studies. Setting: 12 “early adopter” NHS acute hospitals and specialist caresettings studied over two and a half years. Data sources: Data were collected through in depth interviews, observations, and relevant documents relating directly to case study sites and to wider national developments that were perceived to impact on the implementation strategy. Data were thematically analysed, initially within and then across cases. The dataset consisted of 431 semistructured interviews with key stakeholders, including hospital staff, developers, and governmental stakeholders; 590 hours of observations of strategic meetings and use of the software in context; 334 sets of notes from observations, researcher’ field notes, and notes from national conferences; 809 NHS documents; and 58 regional and national documents. Results: Implementation has proceeded more slowly, with a narrower scope and substantially less clinical functionality than was originally planned. The national strategy had considerable local consequences (summarised under five key themes), and wider national developments impacted heavily on implementation and adoption. More specifically, delays related to unrealistic expectations about the capabilities of systems; the time needed to build, configure, and customise the software; the work needed to ensure that systems were supporting provision of care; and the needs of end users for training and support. Other factors hampering progress included the changing milieu of NHS policy and priorities; repeatedly renegotiated national contracts; different stages of development of diverse NHS care records service systems; and a complex communication process between different stakeholders, along with contractual arrangements that largely excluded NHS providers. There was early evidence that deploying systems resulted in important learning within and between organisations and the development of relevant competencies within NHS hospitals. Conclusions: Implementation of the NHS Care Records Service in “early adopter” sites proved time consuming and challenging, with as yet limited discernible benefits for clinicians and no clear advantages for patients. Although our results might not be directly transferable to later adopting sites because the functionalities we evaluated were new and untried in the English context, they shed light on the processes involved in implementing major new systems. The move to increased local decision making that we advocated based on our interim analysis has been pursued and welcomed by the NHS, but it is important that policymakers do not lose sight of the overall goal of an integrated interoperable solution

Conceptualising production, productivity and technology in pharmacy practice: a novel framework for policy, education and research.

CONTEXT AND BACKGROUND: People and health systems worldwide face serious challenges due to shifting disease demographics, rising population demands and weaknesses in healthcare provision, including capacity shortages and lack of impact of healthcare services. These multiple challenges, linked with the global push to achieve universal health coverage, have made apparent the importance of investing in workforce development to improve population health and economic well-being. In relation to medicines, health systems face challenges in terms of access to needed medicines, optimising medicines use and reducing risk. In 2017, the International Pharmaceutical Federation (FIP) published global policy on workforce development ('the Nanjing Statements') that describe an envisioned future for professional education and training. The documents make clear that expanding the pharmacy workforce benefits patients, and continually improving education and training produces better clinical outcomes. AIMS AND PURPOSE: The opportunities for harnessing new technologies in pharmacy practice have been relatively ignored. This paper presents a conceptual framework for analysing production methods, productivity and technology in pharmacy practice that differentiates between dispensing and pharmaceutical care services. We outline a framework that may be employed to study the relationship between pharmacy practice and productivity, shaped by educational and technological inputs. METHOD AND RESULTS: The analysis is performed from the point of view of health systems economics. In relation to pharmaceutical care (patient-oriented practice), pharmacists are service providers; however, their primary purpose is not to deliver consultations, but to maximise the quantum of health gain they secure. Our analysis demonstrates that 'technology shock' is clearly beneficial compared with orthodox notions of productivity or incremental gain implementations. Additionally, the whole process of providing professional services using 'pharmaceutical care technologies' is governed by local institutional frames, suggesting that activities may be structured differently in different places and countries. DISCUSSION AND CONCLUSION: Addressing problems with medication use with the development of a pharmaceutical workforce that is sufficient in quantity and competence is a long-term issue. As a result of this analysis, there emerges a challenge about the profession's relationship with existing and emerging technical innovations. Our novel framework is designed to facilitate policy, education and research by providing an analytical approach to service delivery. By using this approach, the profession could develop examples of good practice in both developed and developing countries worldwide

England’s Electronic Prescription Service: Infrastructure in an Institutional Setting

We describe the development of the Electronic Prescription Service (EPS), the solution for the electronic transmission of prescriptions adopted by the English NHS for primary care. The chapter is based on both an analysis of data collected as part of a nationally commissioned evaluation of EPS, and on reports of contemporary developments in the service. Drawing on the notion of an installed infrastructural base, we illustrate how EPS has been assembled within a rich institutional and organizational context including causal pasts, contemporary practices and policy visions. This process of assembly is traced using three perspectives; as the realization and negotiation of constraints found in the wider NHS context, as a response to inertia arising from limited resources and weak incentive structures, and as a purposive fidelity to the existing institutional cultures of the NHS. The chapter concludes by reflecting on the significance of this analysis for notions of an installed base

Building a house on shifting sand: methodological considerations when evaluating the implementation and adoption of national electronic health record systems

Background: A commitment to Electronic Health Record (EHR) systems now constitutes a core part of many governments’ healthcare reform strategies. The resulting politically-initiated large-scale or national EHR endeavors are challenging because of their ambitious agendas of change, the scale of resources needed to make them work, the (relatively) short timescales set, and the large number of stakeholders involved, all of whom pursue somewhat different interests. These initiatives need to be evaluated to establish if they improve care and represent value for money. Methods: Critical reflections on these complexities in the light of experience of undertaking the first national, longitudinal, and sociotechnical evaluation of the implementation and adoption of England’s National Health Service’s Care Records Service (NHS CRS). Results/discussion: We advance two key arguments. First, national programs for EHR implementations are likely to take place in the shifting sands of evolving sociopolitical and sociotechnical and contexts, which are likely to shape them in significant ways. This poses challenges to conventional evaluation approaches which draw on a model of baseline operations → intervention → changed operations (outcome). Second, evaluation of such programs must account for this changing context by adapting to it. This requires careful and creative choice of ontological, epistemological and methodological assumptions. Summary: New and significant challenges are faced in evaluating national EHR implementation endeavors. Based on experiences from this national evaluation of the implementation and adoption of the NHS CRS in England, we argue for an approach to these evaluations which moves away from seeing EHR systems as Information and Communication Technologies (ICT) projects requiring an essentially outcome-centred assessment towards a more interpretive approach that reflects the situated and evolving nature of EHR seen within multiple specific settings and reflecting a constantly changing milieu of policies, strategies and software, with constant interactions across such boundaries

Video-calls to reduce loneliness and social isolation within care environments for older people: an implementation study using collaborative action research

Background  Older people in care may be lonely with insufficient contact if families are unable to visit. Face-to-face contact through video-calls may help reduce loneliness, but little is known about the processes of engaging people in care environments in using video-calls. We aimed to identify the barriers to and facilitators of implementing video-calls for older people in care environments.  Methods  A collaborative action research (CAR) approach was taken to implement a video-call intervention in care environments. We undertook five steps of recruitment, planning, implementation, reflection and re-evaluation, in seven care homes and one hospital in the UK. The video-call intervention ‘Skype on Wheels’ (SoW) comprised a wheeled device that could hold an iPad and handset, and used Skype to provide a free video-call service. Care staff were collaborators who implemented the intervention within the care-setting by agreeing the intervention, recruiting older people and their family, and setting up video-calls. Field notes and reflective diaries on observations and conversations with staff, older people and family were maintained over 15 months, and analysed using thematic analysis.  Results  Four care homes implemented the intervention. Eight older people with their respective social contacts made use of video-calls. Older people were able to use SoW with assistance from staff, and enjoyed the use of video-calls to stay better connected with family. However five barriers towards implementation included staff turnover, risk averseness, the SoW design, lack of family commitment and staff attitudes regarding technology.  Conclusions  The SoW intervention, or something similar, could aid older people to stay better connected with their families in care environments, but if implemented as part of a rigorous evaluation, then co-production of the intervention at each recruitment site may be needed to overcome barriers and maximise engagement

Journey to DOR: a retro science-fiction story on researching eprescribing

The core of this paper is a science fiction short story. We are on planet DOR. A group of scientists are working on an experiment, changing underlying mechanisms of transmissions of a colossus machine – a complex system of gears and levers, wires and pipes. Some of its mechanisms are also known as D for Doctors, F for Pharmacists, P for Patients. Observers travel from Earth to study the experiment. Their dilemmas are unaided by their advanced research tools. The story is inspired by research carried out for the evaluation of the forthcoming Electronic Prescription Service in England (UK). Our fiction is fiction, but it is also a methodological means, a reflexive lever to elaborate and explore our research texts and to question the feasibility, meaning and impact of researching future technology

Recognising the limits of virtual organizations

No description supplie

Defining the virtual organization

No description supplie