Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

Accelerated placental aging in early onset preeclampsia pregnancies identified by DNA methylation.

Aim: To determine whether dynamic DNA methylation changes in the human placenta can be used to predict gestational age. Materials & methods: Publicly available placental DNA methylation data from 12 studies, together with our own dataset, using Illumina Infinium Human Methylation BeadChip arrays. Results & conclusion: We developed an accurate tool for predicting gestational age of placentas using 62 CpG sites. There was a higher predicted gestational age for placentas from early onset preeclampsia cases, but not term preeclampsia, compared with their chronological age. Therefore, early onset preeclampsia is associated with placental aging. Gestational age acceleration prediction from DNA methylation array data may provide insight into the molecular mechanisms of pregnancy disorders.Benjamin T Mayne, Shalem Y Leemaqz, Alicia K Smith, James Breen, Claire T Roberts, Tina Bianco-Miott

Large scale gene expression meta-analysis reveals tissue-specific, sex-biased gene expression in humans

The severity and prevalence of many diseases are known to differ between the sexes. Organ specific sex-biased gene expression may underpin these and other sexually dimorphic traits. To further our understanding of sex differences in transcriptional regulation, we performed meta-analyses of sex biased gene expression in multiple human tissues. We analyzed 22 publicly available human gene expression microarray data sets including over 2500 samples from 15 different tissues and 9 different organs. Briefly, by using an inverse-variance method we determined the effect size difference of gene expression between males and females. We found the greatest sex differences in gene expression in the brain, specifically in the anterior cingulate cortex, (1818 genes), followed by the heart (375 genes), kidney (224 genes), colon (218 genes), and thyroid (163 genes). More interestingly, we found different parts of the brain with varying numbers and identity of sex-biased genes, indicating that specific cortical regions may influence sexually dimorphic traits. The majority of sex-biased genes in other tissues such as the bladder, liver, lungs, and pancreas were on the sex chromosomes or involved in sex hormone production. On average in each tissue, 32% of autosomal genes that were expressed in a sex-biased fashion contained androgen or estrogen hormone response elements. Interestingly, across all tissues, we found approximately two-thirds of autosomal genes that were sex-biased were not under direct influence of sex hormones. To our knowledge this is the largest analysis of sex-biased gene expression in human tissues to date. We identified many sex-biased genes that were not under the direct influence of sex chromosome genes or sex hormones. These may provide targets for future development of sex-specific treatments for diseases.Benjamin T. Mayne, Tina Bianco-Miotto, Sam Buckberry, James Breen, Vicki Clifton, Cheryl Shoubridge and Claire T. Robert

Exploring the climate of Proxima B with the Met Office Unified Model

This is the author accepted manuscript. The final version is available from EDP Sciences via the DOI in this record.The corrigendum to this article is in ORE at: http://hdl.handle.net/10871/34331We present results of simulations of the climate of the newly discovered planet Proxima Centauri B, performed using the Met Office Unified Model (UM). We examine the responses of both an ‘Earth-like’ atmosphere and simplified nitrogen and trace carbon dioxide atmosphere to the radiation likely received by Proxima Centauri B. Additionally, we explore the effects of orbital eccentricity on the planetary conditions using a range of eccentricities guided by the observational constraints. Overall, our results are in agreement with previous studies in suggesting Proxima Centauri B may well have surface temperatures conducive to the presence of liquid water. Moreover, we have expanded the parameter regime over which the planet may support liquid water to higher values of eccentricity (& 0.1) and lower incident fluxes (881.7 W m−2 ) than previous work. This increased parameter space arises because of the low sensitivity of the planet to changes in stellar flux, a consequence of the stellar spectrum and orbital configuration. However, we also find interesting differences from previous simulations, such as cooler mean surface temperatures for the tidally-locked case. Finally, we have produced high resolution planetary emission and reflectance spectra, and highlight signatures of gases vital to the evolution of complex life on Earth (oxygen, ozone and carbon dioxide).I.B., J.M. and P.E. acknowledge the support of a Met Office Academic Partnership secondment. B.D. thanks the University of Exeter for support through a Ph.D. studentship. N.J.M. and J.G.’s contributions were in part funded by a Leverhulme Trust Research Project Grant, and in part by a University of Exeter College of Engineering, Mathematics and Physical Sciences studentship. We acknowledge use of the MONSooN system, a collaborative facility supplied under the Joint Weather and Climate Research Programme, a strategic partnership between the Met Office and the Natural Environment Research Council. This work also used the University of Exeter Supercomputer, a DiRAC Facility jointly funded by STFC, the Large Facilities Capital Fund of BIS and the University of Exeter

A comparison of GC-FID and PTR-MS toluene measurements in ambient air under conditions of enhanced monoterpene loading

Toluene was measured using both a gas chromatographic system (GC), with a flame ionization detector (FID), and a proton transfer reaction-mass spectrometer (PTR-MS) at the AIRMAP atmospheric monitoring station Thompson Farm (THF) in rural Durham, NH during the summer of 2004. Simultaneous measurements of monoterpenes, including alpha- and beta-pinene, camphene, Delta(3)-carene, and d-limonene, by GC-FID demonstrated large enhancements in monoterpene mixing ratios relative to toluene, with median and maximum enhancement ratios of similar to 2 and similar to 30, respectively. A detailed comparison between the GC-FID and PTR-MS toluene measurements was conducted to test the specificity of PTR-MS for atmospheric toluene measurements under conditions often dominated by biogenic emissions. We derived quantitative estimates of potential interferences in the PTR-MS toluene measurements related to sampling and analysis of monoterpenes, including fragmentation of the monoterpenes and some of their primary carbonyl oxidation products via reactions with H(3)O(+), O(2)(+) and NO(+) in the PTR-MS drift tube. The PTR-MS and GC-FID toluene measurements were in good quantitative agreement and the two systems tracked one another well from the instrumental limits of detection to maximum mixing ratios of similar to 0.5 ppbv. A correlation plot of the PTR-MS versus GC-FID toluene measurements was described by the least squares regression equation y=(1.13 +/- 0.02)x-(0.008 +/- 0.003) ppbv, suggesting a small similar to 13% positive bias in the PTR-MS measurements. The bias corresponded with a similar to 0.055 ppbv difference at the highest measured toluene level. The two systems agreed quantitatively within the combined 1 sigma measurement precisions for 60% of the measurements. Discrepancies in the measured mixing ratios were not well correlated with enhancements in the monoterpenes. Better quantitative agreement between the two systems was obtained by correcting the PTR-MS measurements for contributions from monoterpene fragmentation in the PTR-MS drift tube; however, the improvement was minor (\u3c10%). Interferences in the PTRMS measurements from fragmentation of the monoterpene oxidation products pinonaldehyde, caronaldehyde and alpha-pinene oxide were also likely negligible. A relatively large and variable toluene background in the PTR-MS instrument likely drove the measurement bias; however, the precise contribution was difficult to accurately quantify and thus was not corrected for in this analysis. The results from THF suggest that toluene can be reliably quantified by PTR-MS using our operating conditions (drift tube pressure, temperature and voltage of 2.0 mbar, 45 degrees C and 600V, respectively) under the ambient compositions probed. This work extends the range of field conditions under which PTR-MS validation studies have been conducted

Robust constrained model predictive control based on parameter-dependent Lyapunov functions

The problem of robust constrained model predictive control (MPC) of systems with polytopic uncertainties is considered in this paper. New sufficient conditions for the existence of parameter-dependent Lyapunov functions are proposed in terms of linear matrix inequalities (LMIs), which will reduce the conservativeness resulting from using a single Lyapunov function. At each sampling instant, the corresponding parameter-dependent Lyapunov function is an upper bound for a worst-case objective function, which can be minimized using the LMI convex optimization approach. Based on the solution of optimization at each sampling instant, the corresponding state feedback controller is designed, which can guarantee that the resulting closed-loop system is robustly asymptotically stable. In addition, the feedback controller will meet the specifications for systems with input or output constraints, for all admissible time-varying parameter uncertainties. Numerical examples are presented to demonstrate the effectiveness of the proposed techniques

Effect of anisotropy and destructuration on behavior of Haarajoki test embankment

This paper investigates the influence of anisotropy and destructuration on the behavior of Haarajoki test embankment, which was built by the Finnish National Road Administration as a noise barrier in 1997 on a soft clay deposit. Half of the embankment is constructed on an area improved with prefabricated vertical drains, while the other half is constructed on the natural deposit without any ground improvement. The construction and consolidation of the embankment is analyzed with the finite-element method using three different constitutive models to represent the soft clay. Two recently proposed constitutive models, namely S-CLAY1 which accounts for initial and plastic strain induced anisotropy, and its extension, called S-CLAY1S which accounts, additionally, for interparticle bonding and degradation of bonds, were used in the analysis. For comparison, the problem is also analyzed with the isotropic modified cam clay model. The results of the numerical analyses are compared with the field measurements. The simulations reveal the influence that anisotropy and destructuration have on the behavior of an embankment on soft clay

Realization of quantum process tomography in NMR

Quantum process tomography is a procedure by which the unknown dynamical evolution of an open quantum system can be fully experimentally characterized. We demonstrate explicitly how this procedure can be implemented with a nuclear magnetic resonance quantum computer. This allows us to measure the fidelity of a controlled-not logic gate and to experimentally investigate the error model for our computer. Based on the latter analysis, we test an important assumption underlying nearly all models of quantum error correction, the independence of errors on different qubits.Comment: 8 pages, 7 EPS figures, REVTe

Probing the anomalous extinction of four young star clusters: the use of colour-excess, main sequence fitting and fractal analysis

Four young star clusters were studied in order to characterize their anomalous extinction or variable reddening that could be due to a possible contamination by dense clouds or circumstellar effects. The extinction law (Rv) was evaluated by adopting two methods: (i) the use of theoretical expressions based on the colour-excess of stars with known spectral type, and (ii) the analysis of two-colour diagrams, where the slope of observed colours distribution is compared to the normal distribution. An algorithm to reproduce the zero age main sequence (ZAMS) reddened colours was developed in order to derive the average visual extinction (Av) that provides the best fitting of the observational data. The structure of the clouds was evaluated by means of statistical fractal analysis, aiming to compare their geometric structure with the spatial distribution of the cluster members. The cluster NGC 6530 is the only object of our sample showing anomalous extinction. In average, the other clusters are suffering normal extinction, but several of their members, mainly in NGC 2264, seem to have high Rv, probably due to circumstellar effects. The ZAMS fitting provides Av values that are in good agreement with those found in the literature. The fractal analysis shows that NGC 6530 has a centrally concentrated distribution of stars that is different of the sub-structures found in the density distribution of the cloud projected in the Av map, suggesting that the original cloud has been changed with the cluster formation. On the other hand, the fractal dimension and the statistical parameters of Berkeley 86, NGC 2244, and NGC 2264 indicate a good cloud-cluster correlation, when compared to other works based on artificial distribution of points.Comment: 13 pages, 7 figure