Approximate Newton Methods for Policy Search in Markov Decision Processes

Approximate Newton methods are standard optimization tools which aim to maintain the benefits of Newton's method, such as a fast rate of convergence, while alleviating its drawbacks, such as computationally expensive calculation or estimation of the inverse Hessian. In this work we investigate approximate Newton methods for policy optimization in Markov decision processes (MDPs). We first analyse the structure of the Hessian of the total expected reward, which is a standard objective function for MDPs. We show that, like the gradient, the Hessian exhibits useful structure in the context of MDPs and we use this analysis to motivate two Gauss-Newton methods for MDPs. Like the Gauss- Newton method for non-linear least squares, these methods drop certain terms in the Hessian. The approximate Hessians possess desirable properties, such as negative definiteness, and we demonstrate several important performance guarantees including guaranteed ascent directions, invariance to affine transformation of the parameter space and convergence guarantees. We finally provide a unifying perspective of key policy search algorithms, demonstrating that our second Gauss- Newton algorithm is closely related to both the EM-algorithm and natural gradient ascent applied to MDPs, but performs significantly better in practice on a range of challenging domains

Experimental Heat Transfer Supporting Simulated Water Well Performance on Mars

Favorable indications of massive quantities of water on Mars have initiated studies of potential changes to human Mars missions. Using a technique known as a Rodriguez Well to melt the ice, store the resulting water in a subsurface ice cavity until needed, and then pump water to the surface for use is one potential means to effect these changes. A computer simulation of the Rodriguez Well in a terrestrial environment is one of the engineering tools being used to characterize the performance of this type of well on Mars. An experiment at the NASA Johnson Space Center is gathering data for convective heat transfer and evaporation rates at Mars surface conditions so that this computer simulation can be properly modified to predict performance on Mars. While quantitative results await processing, tests have indicated that a pool of water can be maintained at 1C to 2 C while at Mars surface temperatures and pressures

Si/SiGe bound-to-continuum quantum cascade emitters

Si/SiGe bound-to-continuum quantum cascade emitters designed by self-consistent 6-band k.p modeling and grown by low energy plasma enhanced chemical vapour deposition are presented demonstrating electroluminescence between 1.5 and 3 THz. The electroluminescence is Stark shifted by an electric field and demonstrates polarized emission consistent with the design. Transmission electron microscopy and x-ray diffraction are also presented to characterize the thick heterolayer structure

A scattering rate approach to the understanding of absorption line broadening in near-infrared AlGaN/GaN quantum wells

There has been much interest in the advancement of III-Nitride growth technology to fabricate AlGaN/GaN heterostructures for intersubband transitions (ISBTs). The large conduction band offset in these structures (up to 2 eV) allows transition energies in the near- to the far-infrared region, which have applications from telecommunications, such as in all-optical switches, to infra-red detectors for sensing and imaging. To date, ISBT electroluminescence has been elusive and absorption measurements remain an important method to verify band structure calculations. The growth quality can be inferred from the absorption spectrum, which will have line broadening with contributions that are both inhomogeneous (large-scale interface roughness, and non-parabolicity) and homogeneous (electron scattering related lifetime broadening). In the present work we calculated the contributions of various homogeneous broadening mechanisms (electron interaction with longitudinal-optical (LO) phonons, acoustic phonons, impurities and alloy disorder) to the full linewidth, and also the contribution of band non-parabolicity, which contributes to the inhomogeneous broadening. Calculations are then compared to the measured absorption spectra of several samples

Theory and design of quantum cascade lasers in (111) n-type Si/SiGe

Although most work towards the realization of group IV quantum cascade lasers (QCLs) has focused on valence band transitions, there are many desirable properties associated with the conduction band. We show that the commonly cited shortcomings of n-type Si/SiGe heterostructures can be overcome by moving to the (111) growth direction. Specifically, a large band offset and low effective mass are achievable and subband degeneracy is preserved. We predict net gain up to lattice temperatures of 90 K in a bound-to-continuum QCL with a double-metal waveguide, and show that a Ge interdiffusion length of at least 8 Å across interfaces is tolerable

A High-Resolution Combined Scanning Laser- and Widefield Polarizing Microscope for Imaging at Temperatures from 4 K to 300 K

Polarized light microscopy, as a contrast-enhancing technique for optically anisotropic materials, is a method well suited for the investigation of a wide variety of effects in solid-state physics, as for example birefringence in crystals or the magneto-optical Kerr effect (MOKE). We present a microscopy setup that combines a widefield microscope and a confocal scanning laser microscope with polarization-sensitive detectors. By using a high numerical aperture objective, a spatial resolution of about 240 nm at a wavelength of 405 nm is achieved. The sample is mounted on a $^4$He continuous flow cryostat providing a temperature range between 4 K and 300 K, and electromagnets are used to apply magnetic fields of up to 800 mT with variable in-plane orientation and 20 mT with out-of-plane orientation. Typical applications of the polarizing microscope are the imaging of the in-plane and out-of-plane magnetization via the longitudinal and polar MOKE, imaging of magnetic flux structures in superconductors covered with a magneto-optical indicator film via Faraday effect or imaging of structural features, such as twin-walls in tetragonal SrTiO$_3$. The scanning laser microscope furthermore offers the possibility to gain local information on electric transport properties of a sample by detecting the beam-induced voltage change across a current-biased sample. This combination of magnetic, structural and electric imaging capabilities makes the microscope a viable tool for research in the fields of oxide electronics, spintronics, magnetism and superconductivity.Comment: 14 pages, 11 figures. The following article has been accepted by Review of Scientific Instruments. After it is published, it will be found at http://aip.scitation.org/journal/rs

A Uniformly Derived Catalogue of Exoplanets from Radial Velocities

A new catalogue of extrasolar planets is presented by re-analysing a selection of published radial velocity data sets using EXOFIT (Balan & Lahav 2009). All objects are treated on an equal footing within a Bayesian framework, to give orbital parameters for 94 exoplanetary systems. Model selection (between one- and two-planet solutions) is then performed, using both a visual flagging method and a standard chi-square analysis, with agreement between the two methods for 99% of the systems considered. The catalogue is to be made available online, and this 'proof of concept' study may be maintained and extended in the future to incorporate all systems with publicly available radial velocity data, as well as transit and microlensing data.Comment: 16 pages, 6 figures, 6 table

Electrostatic considerations affecting the calculated HOMO-LUMO gap in protein molecules.

A detailed study of energy differences between the highest occupied and lowest unoccupied molecular orbitals (HOMO-LUMO gaps) in protein systems and water clusters is presented. Recent work questioning the applicability of Kohn-Sham density-functional theory to proteins and large water clusters (E. Rudberg, J. Phys.: Condens. Mat. 2012, 24, 072202) has demonstrated vanishing HOMO-LUMO gaps for these systems, which is generally attributed to the treatment of exchange in the functional used. The present work shows that the vanishing gap is, in fact, an electrostatic artefact of the method used to prepare the system. Practical solutions for ensuring the gap is maintained when the system size is increased are demonstrated. This work has important implications for the use of large-scale density-functional theory in biomolecular systems, particularly in the simulation of photoemission, optical absorption and electronic transport, all of which depend critically on differences between energies of molecular orbitals.Comment: 13 pages, 4 figure

Physicochemical analysis of rotavirus segment 11 supports a 'modified panhandle' structure and not the predicted alternative tRNA-like structure (TRLS)

.Rotaviruses are a major cause of acute gastroenteritis, which is often fatal in infants. The viral genome consists of 11 double-stranded RNA segments, but little is known about their cis-acting sequences and structural elements. Covariation studies and phylogenetic analysis exploring the potential structure of RNA11 of rotaviruses suggested that, besides the previously predicted "modified panhandle" structure, the 5' and 3' termini of one of the isoforms of the bovine rotavirus UKtc strain may interact to form a tRNA-like structure (TRLS). Such TRLSs have been identified in RNAs of plant viruses, where they are important for enhancing replication and packaging. However, using tRNA mimicry assays (in vitro aminoacylation and 3'- adenylation), we found no biochemical evidence for tRNA-like functions of RNA11. Capping, synthetic 3' adenylation and manipulation of divalent cation concentrations did not change this finding. NMR studies on a 5'- and 3'-deletion construct of RNA11 containing the putative intra-strand complementary sequences supported a predominant panhandle structure and did not conform to a cloverleaf fold despite the strong evidence for a predicted structure in this conserved region of the viral RNA. Additional viral or cellular factors may be needed to stabilise it into a form with tRNA-like properties

Food geographies I: relational foodscapes and the busy-ness of being more-than-food

The study of foodscapes has spread throughout geography at the same time as food scholarship has spearheaded post-disciplinary research. This report argues that geographers have taken to post-disciplinarity to explore the ways that food is ‘more-than-food’ through analyses of the visceral nature of eating and politics and the vital (re)materializations of food’s cultural geographies. Visceral food geographies illuminate what I call the ‘contingent relationalities’ of food in the critical evaluation of the indeterminate, situated politics of ‘feeling food’ and those of the embodied collectivities of obesity. Questions remain, however, about how a visceral framework might be deployed for broader critiques within foodscapes and the study of human geography. The study of food’s vital materialisms opens up investigation into the practices of the ‘makings’ of meat, food waste and eating networks. Analysis of affect, embodiment and cultural practices is central to these theorizations and suggests consideration of the multiple materialisms of food, space and eating. There is, I contend, in the more radical, ‘post-relational’ approaches to food, the need for a note of caution. Exuberant claims for the ontological, vital agency of food should be tempered by, or at least run parallel to, critical questions of the real politik of political and practical agency in light of recent struggles over austerity, food poverty and food justice