Retrospective model-based inference guides model-free credit assignment

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants’ momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions

Validity and reliability of the Chinese critical thinking disposition inventory

2004-2005 > Academic research: refereed > Publication in refereed journalVersion of RecordPublishe

Effect of photosynthetically elevated pH on performance of surface flow-constructed wetland planted with Phragmites australis

© 2016, Springer-Verlag Berlin Heidelberg. Combination of emergent and submerged plants has been proved to be able to enhance pollutant removal efficiency of surface flow-constructed wetland (SFCW) during winter. However, intensive photosynthesis of submerged plants during summer would cause pH increase, which may have adverse effects on emergent plants. In this study, nitrogen transformation of lab-scale SFCW under pH gradient of 7.5, 8.5, 9.5 and 10.5 was systematically investigated. The results showed that total nitrogen (TN) removal efficiency decreased from 76.3 ± 0.04 to 51.8 ± 0.04 % when pH increased from 7.5 to 10.5, which was mainly attributed to plant assimilation decay and inhibition of microbe activities (i.e., nitrite-oxidizing bacteria and denitrifiers). Besides, the highest sediment adsorption in SFCW was observed at pH of 8.5. In general, the combination of submerged and emergent plants is feasible for most of the year, but precaution should be taken to mitigate the negative effect of high alkaline conditions when pH rises to above 8.5 in midsummer

The motivational drive to natural rewards is modulated by prenatal glucocorticoid exposure

Exposure to elevated levels of glucocorticoids (GCs) during neurodevelopment has been identified as a triggering factor for the development of reward-associated disorders in adulthood. Disturbances in the neural networks responsible for the complex processes that assign value to rewards and associated stimuli are critical for disorders such as depression, obsessive–compulsive disorders, obesity and addiction. Essential in the understanding on how cues influence behavior is the Pavlovian–instrumental transfer (PIT), a phenomenon that refers to the capacity of a Pavlovian stimulus that predicts a reward to elicit instrumental responses for that same reward. Here, we demonstrate that in utero exposure to GCs (iuGC) impairs both general and selective versions of the PIT paradigm, suggestive of deficits in motivational drive. The iuGC animals presented impaired neuronal activation pattern upon PIT performance in cortical and limbic regions, as well as morphometric changes and reduced levels of dopamine in prefrontal and orbitofrontal cortices, key regions involved in the integration of Pavlovian and instrumental stimuli. Normalization of dopamine levels rescued this behavior, a process that relied on D2/D3, but not D1, dopamine receptor activation. In summary, iuGC exposure programs the mesocorticolimbic dopaminergic circuitry, leading to a reduction in the attribution of the incentive salience to cues, in a dopamine-D2/D3-dependent manner. Ultimately, these results are important to understand how GCs bias incentive processes, a fact that is particularly relevant for disorders where differential attribution of incentive salience is critical.We thank Pedro Morgado for discussions and help in the technical aspects of PIT procedure. This project was supported by a grant of Institute for the Study of Affective Neuroscience (ISAN) and by Janssen Neuroscience Prize. CS-C, SB, MMC and AJR are recipients of Fundacao para a Ciencia e Tecnologia (FCT) fellowships (CS-C: SFRH/BD/51992/2012; SB: SFRH/BD/89936/2012; MMC: SRFH/BD/51061/2010; AJR: SFRH/BPD/33611/2009)

Structural and magnetic phase diagram of CeFeAsO1-xFx and its relationship to high-temperature superconductivity

We use neutron scattering to study the structural and magnetic phase transitions in the iron pnictides CeFeAsO1-xFx as the system is tuned from a semimetal to a high-transition-temperature (high-Tc) superconductor through Fluorine (F) doping x. In the undoped state, CeFeAsO develops a structural lattice distortion followed by a stripe like commensurate antiferromagnetic order with decreasing temperature. With increasing Fluorine doping, the structural phase transition decreases gradually while the antiferromagnetic order is suppressed before the appearance of superconductivity, resulting an electronic phase diagram remarkably similar to that of the high-Tc copper oxides. Comparison of the structural evolution of CeFeAsO1-xFx with other Fe-based superconductors reveals that the effective electronic band width decreases systematically for materials with higher Tc. The results suggest that electron correlation effects are important for the mechanism of high-Tc superconductivity in these Fe pnictides.Comment: 19 pages, 5 figure

Rabies screen reveals GPe control of cocaine-triggered plasticity.

Identification of neural circuit changes that contribute to behavioural plasticity has routinely been conducted on candidate circuits that were preselected on the basis of previous results. Here we present an unbiased method for identifying experience-triggered circuit-level changes in neuronal ensembles in mice. Using rabies virus monosynaptic tracing, we mapped cocaine-induced global changes in inputs onto neurons in the ventral tegmental area. Cocaine increased rabies-labelled inputs from the globus pallidus externus (GPe), a basal ganglia nucleus not previously known to participate in behavioural plasticity triggered by drugs of abuse. We demonstrated that cocaine increased GPe neuron activity, which accounted for the increase in GPe labelling. Inhibition of GPe activity revealed that it contributes to two forms of cocaine-triggered behavioural plasticity, at least in part by disinhibiting dopamine neurons in the ventral tegmental area. These results suggest that rabies-based unbiased screening of changes in input populations can identify previously unappreciated circuit elements that critically support behavioural adaptations

Focus on vulnerable populations and promoting equity in health service utilization ––an analysis of visitor characteristics and service utilization of the Chinese community health service

Background Community health service in China is designed to provide a convenient and affordable primary health service for the city residents, and to promote health equity. Based on data from a large national study of 35 cities across China, we examined the characteristics of the patients and the utilization of community health institutions (CHIs), and assessed the role of community health service in promoting equity in health service utilization for community residents. Methods Multistage sampling method was applied to select 35 cities in China. Four CHIs were randomly chosen in every district of the 35 cities. A total of 88,482 visitors to the selected CHIs were investigated by using intercept survey method at the exit of the CHIs in 2008, 2009, 2010, and 2011. Descriptive analyses were used to analyze the main characteristics (gender, age, and income) of the CHI visitors, and the results were compared with that from the National Health Services Survey (NHSS, including CHIs and higher levels of hospitals). We also analyzed the service utilization and the satisfactions of the CHI visitors. Results The proportions of the children (2.4%) and the elderly (about 22.7%) were lower in our survey than those in NHSS (9.8% and 38.8% respectively). The proportion of the low-income group (26.4%) was apparently higher than that in NHSS (12.5%). The children group had the lowest satisfaction with the CHIs than other age groups. The satisfaction of the low-income visitors was slightly higher than that of the higher-income visitors. The utilization rate of public health services was low in CHIs. Conclusions The CHIs in China appears to fulfill the public health target of uptake by vulnerable populations, and may play an important role in promoting equity in health service utilization. However, services for children and the elderly should be strengthened

Virtual screening for inhibitors of the human TSLP:TSLPR interaction

The pro-inflammatory cytokine thymic stromal lymphopoietin (TSLP) plays a pivotal role in the pathophysiology of various allergy disorders that are mediated by type 2 helper T cell (Th2) responses, such as asthma and atopic dermatitis. TSLP forms a ternary complex with the TSLP receptor (TSLPR) and the interleukin-7-receptor subunit alpha (IL-7Ra), thereby activating a signaling cascade that culminates in the release of pro-inflammatory mediators. In this study, we conducted an in silico characterization of the TSLP: TSLPR complex to investigate the drugability of this complex. Two commercially available fragment libraries were screened computationally for possible inhibitors and a selection of fragments was subsequently tested in vitro. The screening setup consisted of two orthogonal assays measuring TSLP binding to TSLPR: a BLI-based assay and a biochemical assay based on a TSLP: alkaline phosphatase fusion protein. Four fragments pertaining to diverse chemical classes were identified to reduce TSLP: TSLPR complex formation to less than 75% in millimolar concentrations. We have used unbiased molecular dynamics simulations to develop a Markov state model that characterized the binding pathway of the most interesting compound. This work provides a proof-ofprinciple for use of fragments in the inhibition of TSLP: TSLPR complexation

A new framework for cortico-striatal plasticity: behavioural theory meets In vitro data at the reinforcement-action interface

Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s) coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem—action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface

Natural coagulates for wastewater treatment; a review for application and mechanism

The increase of water demand and wastewater generation is among the global concerns in the world. The less effective management of water sources leads to serious consequences, the direct disposal of untreated wastewater is associated with the environmental pollution, elimination of aquatic life and the spread of deadly epidemics. The flocculation process is one of the most important stages in water and wastewater treatment plants, wherein this phase the plankton, colloidal particles, and pollutants are precipitated and removed. Two major types of coagulants are used in the flocculation process included the chemical and natural coagulants. Many studies have been performed to optimize the flocculation process while most of these studies have confirmed the hazardous effects of chemical coagulants utilization on the ecosystem. This chapter reviews a summary of the coagulation/flocculation processes using natural coagulants as well as reviews one of the most effective natural methods of water and wastewater treatment