Using informative behavior to increase engagement while learning from human reward

In this work, we address a relatively unexplored aspect of designing agents that learn from human reward. We investigate how an agent’s non-task behavior can affect a human trainer’s training and agent learning. We use the TAMER framework, which facilitates the training of agents by human-generated reward signals, i.e., judgements of the quality of the agent’s actions, as the foundation for our investigation. Then, starting from the premise that the interaction between the agent and the trainer should be bi-directional, we propose two new training interfaces to increase a human trainer’s active involvement in the training process and thereby improve the agent’s task performance. One provides information on the agent’s uncertainty which is a metric calculated as data coverage, the other on its performance. Our results from a 51-subject user study show that these interfaces can induce the trainers to train longer and give more feedback. The agent’s performance, however, increases only in response to the addition of performance-oriented information, not by sharing uncertainty levels. These results suggest that the organizational maxim about human behavior, “you get what you measure”—i.e., sharing metrics with people causes them to focus on optimizing those metrics while de-emphasizing other objectives—also applies to the training of agents. Using principle component analysis, we show how trainers in the two conditions train agents differently. In addition, by simulating the influence of the agent’s uncertainty–informative behavior on a human’s training behavior, we show that trainers could be distracted by the agent sharing its uncertainty levels about its actions, giving poor feedback for the sake of reducing the agent’s uncertainty without improving the agent’s performance

Genomic-Bioinformatic Analysis of Transcripts Enriched in the Third-Stage Larva of the Parasitic Nematode Ascaris suum

Differential transcription in Ascaris suum was investigated using a genomic-bioinformatic approach. A cDNA archive enriched for molecules in the infective third-stage larva (L3) of A. suum was constructed by suppressive-subtractive hybridization (SSH), and a subset of cDNAs from 3075 clones subjected to microarray analysis using cDNA probes derived from RNA from different developmental stages of A. suum. The cDNAs (n = 498) shown by microarray analysis to be enriched in the L3 were sequenced and subjected to bioinformatic analyses using a semi-automated pipeline (ESTExplorer). Using gene ontology (GO), 235 of these molecules were assigned to ‘biological process’ (n = 68), ‘cellular component’ (n = 50), or ‘molecular function’ (n = 117). Of the 91 clusters assembled, 56 molecules (61.5%) had homologues/orthologues in the free-living nematodes Caenorhabditis elegans and C. briggsae and/or other organisms, whereas 35 (38.5%) had no significant similarity to any sequences available in current gene databases. Transcripts encoding protein kinases, protein phosphatases (and their precursors), and enolases were abundantly represented in the L3 of A. suum, as were molecules involved in cellular processes, such as ubiquitination and proteasome function, gene transcription, protein–protein interactions, and function. In silico analyses inferred the C. elegans orthologues/homologues (n = 50) to be involved in apoptosis and insulin signaling (2%), ATP synthesis (2%), carbon metabolism (6%), fatty acid biosynthesis (2%), gap junction (2%), glucose metabolism (6%), or porphyrin metabolism (2%), although 34 (68%) of them could not be mapped to a specific metabolic pathway. Small numbers of these 50 molecules were predicted to be secreted (10%), anchored (2%), and/or transmembrane (12%) proteins. Functionally, 17 (34%) of them were predicted to be associated with (non-wild-type) RNAi phenotypes in C. elegans, the majority being embryonic lethality (Emb) (13 types; 58.8%), larval arrest (Lva) (23.5%) and larval lethality (Lvl) (47%). A genetic interaction network was predicted for these 17 C. elegans orthologues, revealing highly significant interactions for nine molecules associated with embryonic and larval development (66.9%), information storage and processing (5.1%), cellular processing and signaling (15.2%), metabolism (6.1%), and unknown function (6.7%). The potential roles of these molecules in development are discussed in relation to the known roles of their homologues/orthologues in C. elegans and some other nematodes. The results of the present study provide a basis for future functional genomic studies to elucidate molecular aspects governing larval developmental processes in A. suum and/or the transition to parasitism

Annotation of two large contiguous regions from the Haemonchus contortus genome using RNA-seq and comparative analysis with Caenorhabditis elegans

The genomes of numerous parasitic nematodes are currently being sequenced, but their complexity and size, together with high levels of intra-specific sequence variation and a lack of reference genomes, makes their assembly and annotation a challenging task. Haemonchus contortus is an economically significant parasite of livestock that is widely used for basic research as well as for vaccine development and drug discovery. It is one of many medically and economically important parasites within the strongylid nematode group. This group of parasites has the closest phylogenetic relationship with the model organism Caenorhabditis elegans, making comparative analysis a potentially powerful tool for genome annotation and functional studies. To investigate this hypothesis, we sequenced two contiguous fragments from the H. contortus genome and undertook detailed annotation and comparative analysis with C. elegans. The adult H. contortus transcriptome was sequenced using an Illumina platform and RNA-seq was used to annotate a 409 kb overlapping BAC tiling path relating to the X chromosome and a 181 kb BAC insert relating to chromosome I. In total, 40 genes and 12 putative transposable elements were identified. 97.5% of the annotated genes had detectable homologues in C. elegans of which 60% had putative orthologues, significantly higher than previous analyses based on EST analysis. Gene density appears to be less in H. contortus than in C. elegans, with annotated H. contortus genes being an average of two-to-three times larger than their putative C. elegans orthologues due to a greater intron number and size. Synteny appears high but gene order is generally poorly conserved, although areas of conserved microsynteny are apparent. C. elegans operons appear to be partially conserved in H. contortus. Our findings suggest that a combination of RNA-seq and comparative analysis with C. elegans is a powerful approach for the annotation and analysis of strongylid nematode genomes

Deletion of parasite immune modulatory sequences combined with immune activating signals enhances vaccine mediated protection against filarial nematodes

<p>Background: Filarial nematodes are tissue-dwelling parasites that can be killed by Th2-driven immune effectors, but that have evolved to withstand immune attack and establish chronic infections by suppressing host immunity. As a consequence, the efficacy of a vaccine against filariasis may depend on its capacity to counter parasite-driven immunomodulation.</p> <p>Methodology and Principal Findings: We immunised mice with DNA plasmids expressing functionally-inactivated forms of two immunomodulatory molecules expressed by the filarial parasite Litomosoides sigmodontis: the abundant larval transcript-1 (LsALT) and cysteine protease inhibitor-2 (LsCPI). The mutant proteins enhanced antibody and cytokine responses to live parasite challenge, and led to more leukocyte recruitment to the site of infection than their native forms. The immune response was further enhanced when the antigens were targeted to dendritic cells using a single chain Fv-αDEC205 antibody and co-administered with plasmids that enhance T helper 2 immunity (IL-4) and antigen-presenting cell recruitment (Flt3L, MIP-1α). Mice immunised simultaneously against the mutated forms of LsALT and LsCPI eliminated adult parasites faster and consistently reduced peripheral microfilaraemia. A multifactorial analysis of the immune response revealed that protection was strongly correlated with the production of parasite-specific IgG1 and with the numbers of leukocytes present at the site of infection.</p> <p>Conclusions: We have developed a successful strategy for DNA vaccination against a nematode infection that specifically targets parasite-driven immunosuppression while simultaneously enhancing Th2 immune responses and parasite antigen presentation by dendritic cells.</p&gt

Planck intermediate results. XLI. A map of lensing-induced B-modes

The secondary cosmic microwave background (CMB) $B$-modes stem from the post-decoupling distortion of the polarization $E$-modes due to the gravitational lensing effect of large-scale structures. These lensing-induced $B$-modes constitute both a valuable probe of the dark matter distribution and an important contaminant for the extraction of the primary CMB $B$-modes from inflation. Planck provides accurate nearly all-sky measurements of both the polarization $E$-modes and the integrated mass distribution via the reconstruction of the CMB lensing potential. By combining these two data products, we have produced an all-sky template map of the lensing-induced $B$-modes using a real-space algorithm that minimizes the impact of sky masks. The cross-correlation of this template with an observed (primordial and secondary) $B$-mode map can be used to measure the lensing $B$-mode power spectrum at multipoles up to $2000$. In particular, when cross-correlating with the $B$-mode contribution directly derived from the Planck polarization maps, we obtain lensing-induced $B$-mode power spectrum measurement at a significance level of $12\,\sigma$, which agrees with the theoretical expectation derived from the Planck best-fit $\Lambda$CDM model. This unique nearly all-sky secondary $B$-mode template, which includes the lensing-induced information from intermediate to small ($10\lesssim \ell\lesssim 1000$) angular scales, is delivered as part of the Planck 2015 public data release. It will be particularly useful for experiments searching for primordial $B$-modes, such as BICEP2/Keck Array or LiteBIRD, since it will enable an estimate to be made of the lensing-induced contribution to the measured total CMB $B$-modes.Comment: 20 pages, 12 figures; Accepted for publication in A&A; The B-mode map is part of the PR2-2015 Cosmology Products; available as Lensing Products in the Planck Legacy Archive http://pla.esac.esa.int/pla/#cosmology; and described in the 'Explanatory Supplement' https://wiki.cosmos.esa.int/planckpla2015/index.php/Specially_processed_maps#2015_Lensing-induced_B-mode_ma

Slepian functions and their use in signal estimation and spectral analysis

It is a well-known fact that mathematical functions that are timelimited (or spacelimited) cannot be simultaneously bandlimited (in frequency). Yet the finite precision of measurement and computation unavoidably bandlimits our observation and modeling scientific data, and we often only have access to, or are only interested in, a study area that is temporally or spatially bounded. In the geosciences we may be interested in spectrally modeling a time series defined only on a certain interval, or we may want to characterize a specific geographical area observed using an effectively bandlimited measurement device. It is clear that analyzing and representing scientific data of this kind will be facilitated if a basis of functions can be found that are "spatiospectrally" concentrated, i.e. "localized" in both domains at the same time. Here, we give a theoretical overview of one particular approach to this "concentration" problem, as originally proposed for time series by Slepian and coworkers, in the 1960s. We show how this framework leads to practical algorithms and statistically performant methods for the analysis of signals and their power spectra in one and two dimensions, and on the surface of a sphere.Comment: Submitted to the Handbook of Geomathematics, edited by Willi Freeden, Zuhair M. Nashed and Thomas Sonar, and to be published by Springer Verla

Psychological Determinants of Consumer Acceptance of Personalised Nutrition in 9 European Countries

YesObjective: To develop a model of the psychological factors which predict people’s intention to adopt personalised nutrition. Potential determinants of adoption included perceived risk and benefit, perceived self-efficacy, internal locus of control and health commitment. Methods: A questionnaire, developed from exploratory study data and the existing theoretical literature, and including validated psychological scales was administered to N = 9381 participants from 9 European countries (Germany, Greece, Ireland, Poland, Portugal, Spain, the Netherlands, the UK, and Norway). Results: Structural equation modelling indicated that the greater participants’ perceived benefits to be associated with personalised nutrition, the more positive their attitudes were towards personalised nutrition, and the greater their intention to adopt it. Higher levels of nutrition self-efficacy were related to more positive attitudes towards, and a greater expressed intention to adopt, personalised nutrition. Other constructs positively impacting attitudes towards personalised nutrition included more positive perceptions of the efficacy of regulatory control to protect consumers (e.g. in relation to personal data protection), higher self-reported internal health locus of control, and health commitment. Although higher perceived risk had a negative relationship with attitude and an inverse relationship with perceived benefit, its effects on attitude and intention to adopt personalised nutrition was less influential than perceived benefit. The model was stable across the different European countries, suggesting that psychological factors determining adoption of personalised nutrition have generic applicability across different European countries. Conclusion: The results suggest that transparent provision of information about potential benefits, and protection of consumers’ personal data is important for adoption, delivery of public health benefits, and commercialisation of personalised nutrition.This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement n u 265494 (http://cordis.europa.eu/fp7/home_en.html). Food4Me is the acronym of the project ‘‘Personalised nutrition: an integrated analysis of opportunities and challenges’’ (http://www.food4me.org/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript

Scalar and vector Slepian functions, spherical signal estimation and spectral analysis

It is a well-known fact that mathematical functions that are timelimited (or spacelimited) cannot be simultaneously bandlimited (in frequency). Yet the finite precision of measurement and computation unavoidably bandlimits our observation and modeling scientific data, and we often only have access to, or are only interested in, a study area that is temporally or spatially bounded. In the geosciences we may be interested in spectrally modeling a time series defined only on a certain interval, or we may want to characterize a specific geographical area observed using an effectively bandlimited measurement device. It is clear that analyzing and representing scientific data of this kind will be facilitated if a basis of functions can be found that are "spatiospectrally" concentrated, i.e. "localized" in both domains at the same time. Here, we give a theoretical overview of one particular approach to this "concentration" problem, as originally proposed for time series by Slepian and coworkers, in the 1960s. We show how this framework leads to practical algorithms and statistically performant methods for the analysis of signals and their power spectra in one and two dimensions, and, particularly for applications in the geosciences, for scalar and vectorial signals defined on the surface of a unit sphere.Comment: Submitted to the 2nd Edition of the Handbook of Geomathematics, edited by Willi Freeden, Zuhair M. Nashed and Thomas Sonar, and to be published by Springer Verlag. This is a slightly modified but expanded version of the paper arxiv:0909.5368 that appeared in the 1st Edition of the Handbook, when it was called: Slepian functions and their use in signal estimation and spectral analysi

Efficient preparation of Arabidopsis pollen tubes for ultrastructural analysis using chemical and cryo-fixation

The pollen tube (PT) serves as a model system for investigating plant cell growth and morphogenesis. Ultrastructural studies are indispensable to complement data from physiological and genetic analyses, yet an effective method is lacking for PTs of the model plant Arabidopsis thaliana. Methods: Here, we present reliable approaches for ultrastructural studies of Arabidopsis PTs, as well as an efficient technique for immunogold detection of cell wall epitopes. Using different fixation and embedding strategies, we show the amount of PT ultrastructural details that can be obtained by the different methods. Results: Dozens of cross-sections can be obtained simultaneously by the approach, which facilitates and shortens the time for evaluation. In addition to in vitro-grown PTs, our study follows the route of PTs from germination, growth along the pistil, to the penetration of the dense stylar tissue, which requires considerable mechanical forces. To this end, PTs have different strategies from growing between cells but also between the protoplast and the cell wall and even within each other, where they share a partly common cell wall. The separation of PT cell walls in an outer and an inner layer reported for many plant species is less clear in Arabidopsis PTs, where these cell wall substructures are connected by a distinct transition zone. Conclusions: The major advancement of this method is the effective production of a large number of longitudinal and cross-sections that permits obtaining a detailed and representative picture of pollen tube structures in an unprecedented way. This is particularly important when comparing PTs of wild type and mutants to identify even subtle alterations in cytoarchitecture. Arabidopsis is an excellent plant for genetic manipulation, yet the PTs, several-times smaller compared to tobacco or lily, represent a technical challenge. This study reveals a method to overcome this problem and make Arabidopsis PTs more amenable to a combination of genetic and ultrastructural analyses

Dark Energy Survey year 1 results: Joint analysis of galaxy clustering, galaxy lensing, and CMB lensing two-point functions

We perform a joint analysis of the auto and cross-correlations between three cosmic fields: the galaxy density field, the galaxy weak lensing shear field, and the cosmic microwave background (CMB) weak lensing convergence field. These three fields are measured using roughly 1300 sq. deg. of overlapping optical imaging data from first year observations of the Dark Energy Survey and millimeter-wave observations of the CMB from both the South Pole Telescope Sunyaev-Zel'dovich survey and Planck. We present cosmological constraints from the joint analysis of the two-point correlation functions between galaxy density and galaxy shear with CMB lensing. We test for consistency between these measurements and the DES-only two-point function measurements, finding no evidence for inconsistency in the context of flat $\Lambda$CDM cosmological models. Performing a joint analysis of five of the possible correlation functions between these fields (excluding only the CMB lensing autospectrum) yields $S_{8}\equiv \sigma_8\sqrt{\Omega_{\rm m}/0.3} = 0.782^{+0.019}_{-0.025}$ and $\Omega_{\rm m}=0.260^{+0.029}_{-0.019}$. We test for consistency between these five correlation function measurements and the Planck-only measurement of the CMB lensing autospectrum, again finding no evidence for inconsistency in the context of flat $\Lambda$CDM models. Combining constraints from all six two-point functions yields $S_{8}=0.776^{+0.014}_{-0.021}$ and $\Omega_{\rm m}= 0.271^{+0.022}_{-0.016}$. These results provide a powerful test and confirmation of the results from the first year DES joint-probes analysis