Assessing the payback from health R & D: From ad hoc studies to regular monitoring

Chapter 1 : Introduction • The increasing demands for the benefits of payback from publicly funded R&D to be assessed are based partly on the need to justify or account for expenditure on R&D, and partly on the desire for information to assist resource allocation and the better management of R&D funds. The former consideration is particularly strong in relation to the R&D expenditure that comes out of the wider NHS budget. • In this report a range of categories of payback will be identified along with a variety of methods for assessing them. • The aim of the report is to make recommendations as to how the outcomes from health research might best be monitored on a regular basis. The specific context of the report is the NHS R&D Programme but many of the issues will be relevant for a wide range of funders of health R&D. • The introduction sets out not only a plan of the report but also suggests that readers familiar with the general arguments and existing literature may choose to jump to Chapter 6. Chapter 2 : Review of Existing Approaches to Assessing the Payback from Research • Existing work describes various approaches to valuing research. Some are ex ante and attempt to predict the outcomes of research being considered, others are ex post or retrospective. • The five categories of benefit or payback from health R&D that have been identified involve contributions: to knowledge; to research capacity and future research; to improved information for decision making; to the efficiency, efficacy and equity of health care services; and to the nation’s economic performance. These are shown in Table 1 of the report • The process by which R&D generates final outcomes can be modelled as a sequence. This includes primary outputs such as publications; secondary outputs in the form of policy or administrative decisions; and final outcomes which comprise the health and economic benefits. Feedback loops are also introduced and mitigate the limitations of a linear approach. • Qualitative and quantitative approaches can be used but there are immense problems with time lags and attributing outcomes, and sometimes even outputs, to specific items of research funding. • Four common methods of measuring payback can be used. Expert review, by peers or, sometimes, users is the traditional way of assessing the quality of research. Bibliometric techniques can involve not only counting publications but also using datasets such as the Science Citation Index and Wellcome’s Research Outputs Database (ROD). The various methods of economic analysis of payback are difficult to undertake given the costs and problems of acquiring relevant information and estimating benefits. Social science methods include case studies, which can provide useful information but are resource intensive, and questionnaires to researchers and potential research users. Chapter 3 : Characteristics of a Routine Monitoring System • In moving from ad hoc or research studies of payback towards a more regular monitoring it is noted that whereas there has always been a tradition of evaluation of research, in the public services in general there is now a greater emphasis on audit and performance measurement and indicators. A review of these various systems suggests we should be looking to develop a system of outcomes monitoring that incorporates performance indicators (PIs) and measurement rather than an audit system that is trying to monitor activities against predetermined targets. • Standard characteristics of performance measurement systems do not necessarily apply to research where, for example, there are non-standard outputs. Difficulties have arisen in the USA in attempting to apply the Government Performance and Results Act to research funding agencies. It is shown that because the findings of basic research, in particular, enter a knowledge pool in which people and ideas interact, it is difficult to use a PIs’ approach to track eventual outcomes. However, for some types of health research it has proved more feasible to trace the flow between research outputs and outcomes. • An outcomes monitoring system could be useful if it met the following criteria: relevant to, with as comprehensive coverage as possible of, the funders objectives; relevant to the funder’s decision making processes; encourages accurate compliance; minimises unintended consequences; and has acceptable costs. Chapter 4 : Differences Between Research Types • The range of differences between types of research can be relevant for the design of a routine monitoring system. The OECD distinguishes between basic research, applied research and experimental development. Most DH/NHS research is applied. There might be more of a tradition of publication of findings in applied research in health than in other fields. Nevertheless, the publication and incentives patterns operating in basic research mean that it would be inappropriate to use bibliometric indicators in a simple way across all fields even in health research. • Despite having some differences from health research in publication patterns and in the detailed categories of payback, the broad approach proposed in Chapter 6 could be applied to social care research. • Research that is commissioned, especially by the government, has some of the minimum conditions built into it that are associated with outcomes being generated, in particular because the funder has identified that a contribution in this area will be valuable. Chapter 5 : What Units of Research? • The term programme has various meanings including being used to describe a collection of projects on a common theme and to describe a block of funding for a research unit. • Three main streams or modes of funding can be identified: projects, which are administratively grouped into programmes including a responsive programme; institutions/centres/units; individual researchers. These 3 streams are displayed in Figure 1. It is probable that the regular data-gathering for a monitoring system would operate at the basic level of each stream or mode. • Previous work demonstrates that the full range of benefits can sometimes be applied at the level of projects, either in the responsive mode or in programmes, through the use of questionnaires to researchers. Expert and user review and user surveys have also been applied. • Institutions and centres increasingly have experience not only of traditional periodic expert review but also of producing annual reports, although there are debates about what dimensions to include in such reviews and reports. • Individuals in receipt of research development awards have completed questionnaires during and after the awards. These concentrate on the development of research capacity but can go wider. Chapter 6 : A Possible Comprehensive Outcomes Monitoring System • The proposed system is intended for DH/NHS to monitor the outcomes from its R&D in order to justify the R&D expenditure and assist with managing the portfolio. More detailed information is required for the latter purpose. • We propose a multidimensional approach be adopted to cover all the dimensions of payback and that information be gathered from three sets of sources and Table 3 shows which methods would cover which output/outcome categories. • Firstly, possibly annually, a questionnaire (possibly electronic) covering most payback categories should gather data from the basic level of each funding stream ie. from lead researchers of projects, from research institutions/centres, and from individual award holders. • Secondly, supplementary information should be gathered from external databases (including the citation indices and Wellcome’s ROD). • Thirdly, a range of approaches ie. user surveys, reviews by experts and peers, case studies including economic evaluations, and analysis of sources used in policy documents such as NICE guidelines, would be undertaken on a sample basis. They would provide not only supplementary information but, as with the external databases, would also verify the data collected directly from researchers. • These proposals can be evaluated against the criteria set out in Chapter 3: • The system is relevant to DH’s objectives of generating payback in a range of categories. • Various problems have to be overcome before the system could be fully decision relevant. Firstly it might be necessary to ask researchers to apportion the contribution made to specific outputs from various funding streams. Second, to be decision relevant the information would have to be analysed and presented in a manner consistent with funders’ decision making processes. This would involve a) showing how for each outcome and output, for example publications, data from one project or stream could be compared with those from another and b) demonstrating how different outputs and outcomes could be aggregated. • The questions of accuracy of data, minimisation of unintended consequences and the acceptability of the net costs are also addressed. Chapter 7 : Research and Monitoring • Whilst this report is primarily concerned with moving from ad hoc studies towards a routine monitoring system there are issues that need further research. • Before embarking on full implementation the feasibility needs to be tested of items such as on-line recording of data and asking researchers to attribute proportions of research outputs to separate funding agencies. • Once the system is implemented the value of some items can be better assessed, for example the additional value provided by self reporting of publications beyond that gained from relying on external databases. • The data provided by the system would provide opportunities for further payback research on, for example, links between publications and other categories of payback. • Some items such as network analysis could potentially be added to the monitoring system after further examination of them. • Finally the benefit from the monitoring system itself should be assessed.Department of Health; Wellcome Trus

Making Sense of Blockchain in Food Supply-Chains

Blockchain is a potentially disruptive and game-changing technology that has created excitement about its potential applications. The agriculture industry in New Zealand is facing increased pressure to be able to accurately track and trace their produce in order to provide higher levels of proof to their customers. This study used a q-methodology approach to examine whether blockchain technology can be the solution to these issues and provides recommendations as to what businesses need to in order to make this a reality. The empirical research revealed four distinct groups within the industry; each with different perspectives of blockchain and its potential. Results also found that while industry experts believe blockchain implementation is inevitable and it will solve the current issues, factors such as high set-up costs and the complexity of technology may be inhibitors. Based on these findings, key recommendations on how the industry should proceed in order to overcome these factors that are preventing adoption are derived. Further research is suggested on how the challenges of food safety and security may be overcome with emerging technologies such as Blockchain, IoT and AI

The utilisation of health research in policy-making: Concepts, examples and methods of assessment

The importance of health research utilisation in policy-making, and of understanding the mechanisms involved, is increasingly recognised. Recent reports calling for more resources to improve health in developing countries, and global pressures for accountability, draw greater attention to research-informed policy-making. Key utilisation issues have been described for at least twenty years, but the growing focus on health research systems creates additional dimensions. The utilisation of health research in policy-making should contribute to policies that may eventually lead to desired outcomes, including health gains. In this article, exploration of these issues is combined with a review of various forms of policy-making. When this is linked to analysis of different types of health research, it assists in building a comprehensive account of the diverse meanings of research utilisation. Previous studies report methods and conceptual frameworks that have been applied, if with varying degrees of success, to record utilisation in policy-making. These studies reveal various examples of research impact within a general picture of underutilisation. Factors potentially enhancing utilisation can be identified by exploration of: priority setting; activities of the health research system at the interface between research and policy-making; and the role of the recipients, or 'receptors', of health research. An interfaces and receptors model provides a framework for analysis. Recommendations about possible methods for assessing health research utilisation follow identification of the purposes of such assessments. Our conclusion is that research utilisation can be better understood, and enhanced, by developing assessment methods informed by conceptual analysis and review of previous studies

Helping parents know when to seek help for an acutely ill child: Evidence based co-development of a mobile phone app using complex intervention methodology.

BACKGROUND: Acute illness accounts for the majority of episodes of illness in children under five years of age and is the age group with the highest consultation rate in general practice in the UK. The number of children presenting to emergency care is also steadily increasing, having risen beyond pre-pandemic numbers. Such high, and increasing, rates of consultation have prompted concerns about parents' level of knowledge and confidence in caring for their children when they are ill, and particularly when and how to seek help appropriately. AIM: The ASK SNIFF collaboration research programme identified parents' need for accurate and accessible information to help them know when to seek help for a sick child in 2010. This paper presents the resulting programme of research which aimed to co-develop an evidence-based safety netting intervention (mobile app) to help parents know when to seek help for an acutely ill child under the age of five years in the UK. METHODS: Our programme used a collaborative six step process with 147 parent and 324 health professional participants over a period of six years including: scoping existing interventions, systematic review, qualitative research, video capture, content identification and development, consensus methodology, parent and expert clinical review. RESULTS: Our programme has produced evidence-based content for an app supported by video clips. Our collaborative approach has supported every stage of our work, ensuring that the end result reflects the experiences, perspectives and expressed needs of parents and the clinicians they consult. CONCLUSION: We have not found any other resource which has used this type of approach, which may explain why there is no published evaluation data demonstrating the impact of existing UK resources. Future mobile apps should be designed and developed with the service users for whom they are intended

Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making

The pregenual anterior cingulate cortex (pACC) has been implicated in human anxiety disorders and depression, but the circuit-level mechanisms underlying these disorders are unclear. In healthy individuals, the pACC is involved in cost-benefit evaluation. We developed a macaque version of an approach-avoidance decision task used to evaluate anxiety and depression in humans and, with multi-electrode recording and cortical microstimulation, we probed pACC function as monkeys performed this task. We found that the macaque pACC has an opponent process-like organization of neurons representing motivationally positive and negative subjective value. Spatial distribution of these two neuronal populations overlapped in the pACC, except in one subzone, where neurons with negative coding were more numerous. Notably, microstimulation in this subzone, but not elsewhere in the pACC, increased negative decision-making, and this negative biasing was blocked by anti-anxiety drug treatment. This cortical zone could be critical for regulating negative emotional valence and anxiety in decision-making.National Institutes of Health (U.S.) (Javits Merit Grant R01 NS025529)United States. Office of Naval Research (N000140710903)National Parkinson Foundation (U.S.) (Lynn Diamond Fellowship

Connectivity-based parcellation of the human frontal polar cortex

The frontal pole corresponds to Brodmann area (BA) 10, the largest single architectonic area in the human frontal lobe. Generally, BA10 is thought to contain two or three subregions that subserve broad functions such as multitasking, social cognition, attention, and episodic memory. However, there is a substantial debate about the functional and structural heterogeneity of this large frontal region. Previous connectivity-based parcellation studies have identified two or three subregions in the human frontal pole. Here, we used diffusion tensor imaging to assess structural connectivity of BA10 in 35 healthy subjects and delineated subregions based on this connectivity. This allowed us to determine the correspondence of structurally based subregions with the scheme previously defined functionally. Three subregions could be defined in each subject. However, these three subregions were not spatially consistent between subjects. Therefore, we accepted a solution with two subregions that encompassed the lateral and medial frontal pole. We then examined resting-state functional connectivity of the two subregions and found significant differences between their connectivities. The medial cluster was connected to nodes of the default-mode network, which is implicated in internally focused, self-related thought, and social cognition. The lateral cluster was connected to nodes of the executive control network, associated with directed attention and working memory. These findings support the concept that there are two major anatomical subregions of the frontal pole related to differences in functional connectivity

Challenges and strategies for general practitioners diagnosing serious infections in older adults: a UK qualitative interview study.

BACKGROUND Serious infections in older people are associated with unplanned hospital admissions and high mortality. Recognising the presence of a serious infection and making an accurate diagnosis are important challenges for General Practice. This study aimed to explore the issues UK GPs face when diagnosing serious infections in older patients. METHODS Qualitative study using semi-structured interviews. 28 GPs from 27 practices were purposively sampled from across the UK to achieve maximum variation in terms of GP role, experience and practice population. Interviews began by asking participants to describe recent or memorable cases where they had assessed older patients with suspected serious infections. Additional questions from the topic guide were used to explore the challenges further. Interview transcripts were coded and analysed using a modified framework approach. RESULTS Diagnosing serious infection in older adults was perceived to be challenging by participating GPs and the diagnosis was often uncertain. Contributing factors included patient complexity, atypical presentations, as well as a lack of knowledge of patients due to a loss in continuity. Diagnostic challenges were present at each stage of the patient assessment. Scoring systems were mainly used as communication tools. Investigations were sometimes used to resolve diagnostic uncertainty, but availability and speed of result limited their practical use. Clear safety-net plans shared with patients and their families helped GPs manage ongoing uncertainty. CONCLUSIONS Diagnostic challenges are present throughout the assessment of an older adult with a serious infection in primary care. Supporting GPs to provide continuity of care may improve the recognition and developing point of care testing for use in community settings may reduce diagnostic uncertainty

Effort-Based Cost-Benefit Valuation and the Human Brain

In both the wild and the laboratory, animals' preferences for one course of action over another reflect not just reward expectations but also the cost in terms of effort that must be invested in pursuing the course of action. The ventral striatum and dorsal anterior cingulate cortex (ACCd) are implicated in the making of cost-benefit decisions in the rat, but there is little information about how effort costs are processed and influence calculations of expected net value in other mammals including humans. We performed a functional magnetic resonance imaging study to determine whether and where activity in the human brain was available to guide effort-based cost-benefit valuation. Subjects were scanned while they performed a series of effortful actions to obtain secondary reinforcers. At the beginning of each trial, subjects were presented with one of eight different visual cues that they had learned indicated how much effort the course of action would entail and how much reward could be expected at its completion. Cue-locked activity in the ventral striatum and midbrain reflected the net value of the course of action, signaling the expected amount of reward discounted by the amount of effort to be invested. Activity in ACCd also reflected the interaction of both expected reward and effort costs. Posterior orbitofrontal and insular activity, however, only reflected the expected reward magnitude. The ventral striatum and anterior cingulate cortex may be the substrate of effort-based cost-benefit valuation in primates as well as in rats

Recurrence of Cardiovascular Events in Patients With Type 2 Diabetes : Epidemiology and risk factors

OBJECTIVE—The purpose of this study was to assess incidence of and risk factors for recurrent cardiovascular disease (CVD) in type 2 diabetes

Global Distribution of Rubella Virus Genotypes

Phylogenetic analysis of a collection of 103 E1 gene sequences from rubella viruses isolated from 17 countries from 1961 to 2000 confirmed the existence of at least two genotypes. Rubella genotype I (RGI) isolates, predominant in Europe, Japan, and the Western Hemisphere, segregated into discrete subgenotypes; intercontinental subgenotypes present in the 1960s and 1970s were replaced by geographically restricted subgenotypes after ~1980. Recently, active subgenotypes include one in the United States and Latin America, one in China, and a third that apparently originated in Asia and spread to Europe and North America, starting in 1997, indicating the recent emergence of an intercontinental subgenotype. A virus that potentially arose as a recombinant between two RGI subgenotypes was discovered. Rubella genotype II (RGII) showed greater genetic diversity than did RGI and may actually consist of multiple genotypes. RGII viruses were limited to Asia and Europe; RGI viruses were also present in most of the countries where RGII viruses were isolated