Bandit Models of Human Behavior: Reward Processing in Mental Disorders

Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for multi-armed bandit problem, which extends the standard Thompson Sampling approach to incorporate reward processing biases associated with several neurological and psychiatric conditions, including Parkinson's and Alzheimer's diseases, attention-deficit/hyperactivity disorder (ADHD), addiction, and chronic pain. We demonstrate empirically that the proposed parametric approach can often outperform the baseline Thompson Sampling on a variety of datasets. Moreover, from the behavioral modeling perspective, our parametric framework can be viewed as a first step towards a unifying computational model capturing reward processing abnormalities across multiple mental conditions.Comment: Conference on Artificial General Intelligence, AGI-1

Exercise-based cardiac rehabilitation for coronary heart disease

PublishedReviewThis review is published as a Cochrane Review in the Cochrane Database of Systematic Reviews 2016, Issue 1. Cochrane Reviews are regularly updated as new evidence emerges and in response to comments and criticisms, and the Cochrane Database of Systematic Reviews should be consulted for the most recent version of the Review.Background Coronary heart disease (CHD) is the single most common cause of death globally. However, with falling CHD mortality rates, an increasing number of people live with CHD and may need support to manage their symptoms and prognosis. Exercise-based cardiac rehabilitation (CR) aims to improve the health and outcomes of people with CHD. This is an update of a Cochrane systematic review previously published in 2011. Objectives To assess the effectiveness and cost-effectiveness of exercise-based CR (exercise training alone or in combination with psychosocial or educational interventions) compared with usual care on mortality, morbidity and HRQL in patients with CHD. To explore the potential study level predictors of the effectiveness of exercise-based CR in patients with CHD. Search methods We updated searches from the previous Cochrane review, by searching Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library, Issue 6, 2014) from December 2009 to July 2014. We also searched MEDLINE (Ovid), EMBASE (Ovid), CINAHL (EBSCO) and Science Citation Index Expanded (December 2009 to July 2014). Selection criteria We included randomised controlled trials (RCTs) of exercise-based interventions with at least six months’ follow-up, compared with a no exercise control. The study population comprised men and women of all ages who have had a myocardial infarction (MI), coronary artery bypass graft (CABG) or percutaneous coronary intervention (PCI), or who have angina pectoris, or coronary artery disease. We included RCTs that reported at least one of the following outcomes: mortality, MI, revascularisations, hospitalisations, health-related quality of life (HRQL), or costs. Data collection and analysis Two review authors independently screened all identified references for inclusion based on the above inclusion and exclusion criteria. One author extracted data from the included trials and assessed their risk of bias; a second review author checked data. We stratified meta-analysis by the duration of follow up of trials, i.e. short-term: 6 to 12 months, medium-term: 13 to 36 months, and long-term: > 3 years. Main results This review included 63 trials which randomised 14,486 people with CHD. This latest update identified 16 new trials (3872 participants). The population included predominantly post-MI and post-revascularisation patients and the mean age of patients within the trials ranged from 47.5 to 71.0 years.Women accounted for fewer than 15% of the patients recruited. Overall trial reporting was poor, although there was evidence of an improvement in quality of reporting in more recent trials. As we found no significant difference in the impact of exercise-based CR on clinical outcomes across follow-up, we focused on reporting findings pooled across all trials at their longest follow-up (median 12 months). Exercise-based CR reduced cardiovascular mortality compared with no exercise control (27 trials; risk ratio (RR) 0.74, 95% CI 0.64 to 0.86). There was no reduction in total mortality with CR (47 trials, RR 0.96, 95% CI 0.88 to 1.04). The overall risk of hospital admissions was reduced with CR (15 trials; RR 0.82, 95% CI 0.70 to 0.96) but there was no significant impact on the risk of MI (36 trials; RR 0.90, 95% CI 0.79 to 1.04), CABG (29 trials; RR 0.96, 95% CI 0.80 to 1.16) or PCI (18 trials; RR 0.85, 95% CI 0.70 to 1.04). There was little evidence of statistical heterogeneity across trials for all event outcomes, and there was evidence of small study bias for MI and hospitalisation, but no other outcome. Predictors of clinical outcomes were examined across the longest follow-up of studies using univariate meta-regression. Results show that benefits in outcomes were independent of participants’ CHD case mix (proportion of patients with MI), type of CR (exercise only vs comprehensive rehabilitation) dose of exercise, length of follow-up, trial publication date, setting (centre vs home-based), study location (continent), sample size or risk of bias. Given the heterogeneity in outcome measures and reporting methods, meta-analysis was not undertaken for HRQL. In five out of 20 trials reporting HRQL using validated measures, there was evidence of significant improvement in most or all of the sub-scales with exercise-based CR compared to control at follow-up. Four trial-based economic evaluation studies indicated exercise-based CR to be a potentially cost-effective use of resources in terms of gain in quality-adjusted life years. The quality of the evidence for outcomes reported in the review was rated using the GRADE method. The quality of the evidence varied widely by outcome and ranged from low to moderate. Authors’ conclusions This updated Cochrane review supports the conclusions of the previous version of this review that, compared with no exercise control, exercise-based CR reduces the risk of cardiovascular mortality but not total mortality. We saw a significant reduction in the risk of hospitalisation with CR but not in the risk of MI or revascularisation. We identified further evidence supporting improved HRQL with exercise-based CR. More recent trials were more likely to be well reported and include older and female patients. However, the population studied in this review still consists predominantly of lower risk individuals following MI or revascularisation. Further well conducted RCTs are needed to assess the impact of exercise-based CR in higher risk CHD groups and also those presenting with stable angina. These trials should include validated HRQL outcome measures, explicitly report clinical event outcomes including mortality and hospital admissions, and assess costs and cost-effectiveness

Haptics for the development of fundamental rhythm skills, including multi-limb coordination

This chapter considers the use of haptics for learning fundamental rhythm skills, including skills that depend on multi-limb coordination. Different sensory modalities have different strengths and weaknesses for the development of skills related to rhythm. For example, vision has low temporal resolution and performs poorly for tracking rhythms in real-time, whereas hearing is highly accurate. However, in the case of multi-limbed rhythms, neither hearing nor sight are particularly well suited to communicating exactly which limb does what and when, or how the limbs coordinate. By contrast, haptics can work especially well in this area, by applying haptic signals independently to each limb. We review relevant theories, including embodied interaction and biological entrainment. We present a range of applications of the Haptic Bracelets, which are computer-controlled wireless vibrotactile devices, one attached to each wrist and ankle. Haptic pulses are used to guide users in playing rhythmic patterns that require multi-limb coordination. One immediate aim of the system is to support the development of practical rhythm skills and multi-limb coordination. A longer-term goal is to aid the development of a wider range of fundamental rhythm skills including recognising, identifying, memorising, retaining, analysing, reproducing, coordinating, modifying and creating rhythms – particularly multi-stream (i.e. polyphonic) rhythmic sequences. Empirical results are presented. We reflect on related work, and discuss design issues for using haptics to support rhythm skills. Skills of this kind are essential not just to drummers and percussionists but also to keyboards players, and more generally to all musicians who need a firm grasp of rhythm

Seasonal changes in patterns of gene expression in avian song control brain regions.

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Photoperiod and hormonal cues drive dramatic seasonal changes in structure and function of the avian song control system. Little is known, however, about the patterns of gene expression associated with seasonal changes. Here we address this issue by altering the hormonal and photoperiodic conditions in seasonally-breeding Gambel's white-crowned sparrows and extracting RNA from the telencephalic song control nuclei HVC and RA across multiple time points that capture different stages of growth and regression. We chose HVC and RA because while both nuclei change in volume across seasons, the cellular mechanisms underlying these changes differ. We thus hypothesized that different genes would be expressed between HVC and RA. We tested this by using the extracted RNA to perform a cDNA microarray hybridization developed by the SoNG initiative. We then validated these results using qRT-PCR. We found that 363 genes varied by more than 1.5 fold (>log(2) 0.585) in expression in HVC and/or RA. Supporting our hypothesis, only 59 of these 363 genes were found to vary in both nuclei, while 132 gene expression changes were HVC specific and 172 were RA specific. We then assigned many of these genes to functional categories relevant to the different mechanisms underlying seasonal change in HVC and RA, including neurogenesis, apoptosis, cell growth, dendrite arborization and axonal growth, angiogenesis, endocrinology, growth factors, and electrophysiology. This revealed categorical differences in the kinds of genes regulated in HVC and RA. These results show that different molecular programs underlie seasonal changes in HVC and RA, and that gene expression is time specific across different reproductive conditions. Our results provide insights into the complex molecular pathways that underlie adult neural plasticity

Developments in CLARA accelerator design and simulations

We present recent developments in the accelerator design of CLARA (Compact Linear Accelerator for Research and Applications), the proposed UK FEL test facility at Daresbury Laboratory. Updates on the electron beam simulations and code comparisons including wakefields are described. Simulations of the effects of geometric wakefields in the small-aperture FEL undulator are shown, as well as further simulations on potential FEL experiments using chirped beams. We also present the results of simulations on post-FEL diagnostics

Electrostatically gated membrane permeability in inorganic protocells

Although several strategies are now available to produce functional microcompartments analogous to primitive cell-like structures, little progress has been made in generating protocell constructs with self-controlled membrane permeability. Here we describe the preparation of water-dispersible colloidosomes based on silica nanoparticles and delineated by a continuous semipermeable inorganic membrane capable of self-activated, electrostatically gated permeability. We use crosslinking and covalent grafting of a pH-responsive copolymer to generate an ultrathin elastic membrane that exhibits selective release and uptake of small molecules. This behaviour, which depends on the charge of the copolymer coronal layer, serves to trigger enzymatic dephosphorylation reactions specifically within the protocell aqueous interior. This system represents a step towards the design and construction of alternative types of artificial chemical cells and protocell models based on spontaneous processes of inorganic self-organization

Cover to Volume 3

The fibroblast mitogen platelet-derived growth factor -BB (PDGF-BB) induces a transient expression of the orphan nuclear receptor NR4A1 (also named Nur77, TR3 or NGFIB). The aim of the present study was to investigate the pathways through which NR4A1 is induced by PDGF-BB and its functional role. We demonstrate that in PDGF-BB stimulated NIH3T3 cells, the MEK1/2 inhibitor CI-1040 strongly represses NR4A1 expression, whereas Erk5 downregulation delays the expression, but does not block it. Moreover, we report that treatment with the NF-κB inhibitor BAY11-7082 suppresses NR4A1 mRNA and protein expression. The majority of NR4A1 in NIH3T3 was found to be localized in the cytoplasm and only a fraction was translocated to the nucleus after continued PDGF-BB treatment. Silencing NR4A1 slightly increased the proliferation rate of NIH3T3 cells; however, it did not affect the chemotactic or survival abilities conferred by PDGF-BB. Moreover, overexpression of NR4A1 promoted anchorage-independent growth of NIH3T3 cells and the glioblastoma cell lines U-105MG and U-251MG. Thus, whereas NR4A1, induced by PDGF-BB, suppresses cell growth on a solid surface, it increases anchorage-independent growth

Passive phloem loading and long-distance transport in a synthetic tree-on-a-chip

Vascular plants rely on differences of osmotic pressure to export sugars from regions of synthesis (mature leaves) to sugar sinks (roots, fruits). In this process, known as M\"unch pressure flow, the loading of sugars from photosynthetic cells to the export conduit (the phloem) is crucial, as it sets the pressure head necessary to power long-distance transport. Whereas most herbaceous plants use active mechanisms to increase phloem concentration above that of the photosynthetic cells, in most tree species, for which transport distances are largest, loading seems to occur via passive symplastic diffusion from the mesophyll to the phloem. Here, we use a synthetic microfluidic model of a passive loader to explore the nonlinear dynamics that arise during export and determine the ability of passive loading to drive long-distance transport. We first demonstrate that in our device, phloem concentration is set by the balance between the resistances to diffusive loading from the source and convective export through the phloem. Convection-limited export corresponds to classical models of M\"unch transport, where phloem concentration is close to that of the source; in contrast, diffusion-limited export leads to small phloem concentrations and weak scaling of flow rates with the hydraulic resistance. We then show that the effective regime of convection-limited export is predominant in plants with large transport resistances and low xylem pressures. Moreover, hydrostatic pressures developed in our synthetic passive loader can reach botanically relevant values as high as 10 bars. We conclude that passive loading is sufficient to drive long-distance transport in large plants, and that trees are well suited to take full advantage of passive phloem loading strategies

Imaging cervical cytology with scanning near-field optical microscopy (SNOM) coupled with an IR-FEL

Cervical cancer remains a major cause of morbidity and mortality among women, especially in the developing world. Increased synthesis of proteins, lipids and nucleic acids is a pre-condition for the rapid proliferation of cancer cells. We show that scanning near-field optical microscopy, in combination with an infrared free electron laser (SNOM-IR-FEL), is able to distinguish between normal and squamous low-grade and high-grade dyskaryosis, and between normal and mixed squamous/glandular pre-invasive and adenocarcinoma cervical lesions, at designated wavelengths associated with DNA, Amide I/II and lipids. These findings evidence the promise of the SNOM-IR-FEL technique in obtaining chemical information relevant to the detection of cervical cell abnormalities and cancer diagnosis at spatial resolutions below the diffraction limit (?0.2 \ensuremathμm). We compare these results with analyses following attenuated total reflection Fourier-transform infrared (ATR-FTIR) spectroscopy; although this latter approach has been demonstrated to detect underlying cervical atypia missed by conventional cytology, it is limited by a spatial resolution of ~3 \ensuremathμm to 30 \ensuremathμm due to the optical diffraction limit