Hi-Val: Iterative Learning of Hierarchical Value Functions for Policy Generation

Task decomposition is effective in manifold applications where the global complexity of a problem makes planning and decision-making too demanding. This is true, for example, in high-dimensional robotics domains, where (1) unpredictabilities and modeling limitations typically prevent the manual specification of robust behaviors, and (2) learning an action policy is challenging due to the curse of dimensionality. In this work, we borrow the concept of Hierarchical Task Networks (HTNs) to decompose the learning procedure, and we exploit Upper Confidence Tree (UCT) search to introduce HOP, a novel iterative algorithm for hierarchical optimistic planning with learned value functions. To obtain better generalization and generate policies, HOP simultaneously learns and uses action values. These are used to formalize constraints within the search space and to reduce the dimensionality of the problem. We evaluate our algorithm both on a fetching task using a simulated 7-DOF KUKA light weight arm and, on a pick and delivery task with a Pioneer robot

Experimental Results of Concurrent Learning Adaptive Controllers

Commonly used Proportional-Integral-Derivative based UAV flight controllers are often seen to provide adequate trajectory-tracking performance only after extensive tuning. The gains of these controllers are tuned to particular platforms, which makes transferring controllers from one UAV to other time-intensive. This paper suggests the use of adaptive controllers in speeding up the process of extracting good control performance from new UAVs. In particular, it is shown that a concurrent learning adaptive controller improves the trajectory tracking performance of a quadrotor with baseline linear controller directly imported from another quadrotors whose inertial characteristics and throttle mapping are very di fferent. Concurrent learning adaptive control uses specifi cally selected and online recorded data concurrently with instantaneous data and is capable of guaranteeing tracking error and weight error convergence without requiring persistency of excitation. Flight-test results are presented on indoor quadrotor platforms operated in MIT's RAVEN environment. These results indicate the feasibility of rapidly developing high-performance UAV controllers by using adaptive control to augment a controller transferred from another UAV with similar control assignment structure.United States. Office of Naval Research. Multidisciplinary University Research Initiative (Grant N000141110688)National Science Foundation (U.S.). Graduate Research Fellowship Program (Grant 0645960)Boeing Scientific Research Laboratorie

Systematic Relationships Between Lidar Observables and Sizes And Mineral Composition Of Dust Aerosols

The physical and chemical properties of soil dust aerosol particles fundamentally affect their interaction with climate, including shortwave absorption and radiative forcing, nucleation of cloud droplets and ice crystals, heterogeneous formation of sulfates and nitrates on the surface of dust particles, and atmospheric processing of iron into bioavailable forms that increase the productivity of marine phytoplankton. Lidar measurements, such as extinction-to-backscatter, color and depolarization ratios, are frequently used to distinguish between aerosol types with different physical and chemical properties. The chemical composition of aerosol particles determines their complex refractive index, hence affecting their backscattering properties. Here we present a study on how dust aerosol backscattering and depolarization properties at wavelengths of 355, 532 and 1064 nm are related to size and complex refractive index, which varies with the mineral composition of the dust. Dust aerosols are represented by collections of spheroids with a range of prolate and oblate aspect ratios and their optical properties are obtained using T-matrix calculations. We find simple, systematic relationships between lidar observables and the dust size and complex refractive index that may aid the use of space-based or airborne lidars for direct retrieval of dust properties or for the evaluation of chemical transport models using forward simulated lidar variables. In addition, we present first results on the spatial variation of forward-simulated lidar variables based on a dust model that accounts for the atmospheric cycle of eight different mineral types plus internal mixtures of seven mineral types with iron oxides, which was recently implemented in the NASA GISS Earth System ModelE2

Actuator Constrained Trajectory Generation and Control for Variable-Pitch Quadrotors

Control and trajectory generation algorithms for a quadrotor helicopter with variable-pitch propellers are presented. The control law is not based on near-hover assumptions, allowing for large attitude deviations from hover. The trajectory generation algorithm ts a time-parametrized polynomial through any number of way points in R3, with a closed-form solution if the corresponding way point arrival times are known a priori. When time is not specifi ed, an algorithm for fi nding minimum-time paths subject to hardware actuator saturation limitations is presented. Attitude-specifi c constraints are easily embedded in the polynomial path formulation, allowing for aerobatic maneuvers to be performed using a single controller and trajectory generation algorithm. Experimental results on a variable pitch quadrotor demonstrate the control design and example trajectories.National Science Foundation (U.S.) (Graduate Research Fellowship under Grant No. 0645960

Inherent-Structure Dynamics and Diffusion in Liquids

The self-diffusion constant D is expressed in terms of transitions among the local minima of the potential (inherent structure, IS) and their correlations. The formulae are evaluated and tested against simulation in the supercooled, unit-density Lennard-Jones liquid. The approximation of uncorrelated IS-transition (IST) vectors, D_{0}, greatly exceeds D in the upper temperature range, but merges with simulation at reduced T ~ 0.50. Since uncorrelated IST are associated with a hopping mechanism, the condition D ~ D_{0} provides a new way to identify the crossover to hopping. The results suggest that theories of diffusion in deeply supercooled liquids may be based on weakly correlated IST.Comment: submitted to PR

The Potential Energy Landscape and Mechanisms of Diffusion in Liquids

The mechanism of diffusion in supercooled liquids is investigated from the potential energy landscape point of view, with emphasis on the crossover from high- to low-T dynamics. Molecular dynamics simulations with a time dependent mapping to the associated local mininum or inherent structure (IS) are performed on unit-density Lennard-Jones (LJ). New dynamical quantities introduced include r2_{is}(t), the mean-square displacement (MSD) within a basin of attraction of an IS, R2(t), the MSD of the IS itself, and g_{loc}(t) the mean waiting time in a cooperative region. At intermediate T, r2_{is}(t) posesses an interval of linear t-dependence allowing calculation of an intrabasin diffusion constant D_{is}. Near T_{c} diffusion is intrabasin dominated with D = D_{is}. Below T_{c} the local waiting time tau_{loc} exceeds the time, tau_{pl}, needed for the system to explore the basin, indicating the action of barriers. The distinction between motion among the IS below T_{c} and saddle, or border dynamics above T_{c} is discussed.Comment: submitted to pr

A critical assessment of methods for the intrinsic analysis of liquid interfaces. 1. surface site distributions

Substantial progress in our understanding of interfacial structure and dynamics has stemmed from the recent development of algorithms that allow for an intrinsic analysis of fluid interfaces. These work by identifying the instantaneous location of the interface, at the atomic level, for each molecular configuration and then computing properties relative to this location. Such a procedure eliminates the broadening of the interface caused by capillary waves and reveals the underlying features of the system. However, a precise definition of which molecules actually belong to the interfacial layer is difficult to achieve in practice. Furthermore, it is not known if the different intrinsic analysis methods are consistent with each other and yield similar results for the interfacial properties. In this paper, we carry out a systematic and detailed comparison of the available methods for intrinsic analysis of fluid interfaces, based on a molecular dynamics simulation of the interface between liquid water and carbon tetrachloride. We critically assess the advantages and shortcomings of each method, based on reliability, robustness, and speed of computation, and establish consistent criteria for determining which molecules belong to the surface layer. We believe this will significantly contribute to make intrinsic analysis methods widely and routinely applicable to interfacial systems

Potential energy landscape-based extended van der Waals equation

The inherent structures ({\it IS}) are the local minima of the potential energy surface or landscape, $U({\bf r})$, of an {\it N} atom system. Stillinger has given an exact {\it IS} formulation of thermodynamics. Here the implications for the equation of state are investigated. It is shown that the van der Waals ({\it vdW}) equation, with density-dependent $a$ and $b$ coefficients, holds on the high-temperature plateau of the averaged {\it IS} energy. However, an additional ``landscape'' contribution to the pressure is found at lower $T$. The resulting extended {\it vdW} equation, unlike the original, is capable of yielding a water-like density anomaly, flat isotherms in the coexistence region {\it vs} {\it vdW} loops, and several other desirable features. The plateau energy, the width of the distribution of {\it IS}, and the ``top of the landscape'' temperature are simulated over a broad reduced density range, $2.0 \ge \rho \ge 0.20$, in the Lennard-Jones fluid. Fits to the data yield an explicit equation of state, which is argued to be useful at high density; it nevertheless reproduces the known values of $a$ and $b$ at the critical point

Increased yield of smear positive pulmonary TB cases by screening patients with >2 weeks cough, compared to >3 weeks and adequacy of 2 sputum smear examinations for diagnosis

Background RNTCP recommends examining three sputum smears for AFB from Chest Symptomatics (CSs) with cough of >3 weeks for diagnosis of Pulmonary TB (PTB). A previous multi-centric study from Tuberculosis Research centre (TRC) has shown that the yield of sputum positive cases can be increased if duration of cough for screening was reduced to >2 weeks. Other studies have shown that two smear examinations are adequate for diagnosis of smear positive PTB . To validate the above findings, a cross sectional multi-centric study was repeated in different settings in five geographical areas in India. Methods Three primary and secondary level health facilities with high out-patient attendance were selected from two Tuberculosis Units (TU) in each of the 15 selected districts to screen about 10,000 new adult outpatients from each state. For patients who did not volunteer history of cough, symptoms were elicited using a structured simple questionnaire. All the CSs were referred for sputum examination. Results A total of 96,787 out-patients were registered. Among them 69,209 (72%) were new adult out-patients. Using >2 weeks of cough instead of > 3 weeks as the criterion for screening, there was an overall increase of 58% in CS and 23% increase in the detection of smear-positive cases. Among 211 patients, 210 were positive at least by one smear from the initial two specimens. Increase in the work-load if 2 smears were done for patients with cough of >2 weeks cough were 2 specimens (i.e.13 to 15) per day for an adult OPD of 150. Conclusion The yield of sputum positive PTB cases can be improved by screening patients with >2 weeks cough and two specimens are adequate for diagnosis

Evolution of honesty in higher-order social networks

Sender-receiver games are simple models of information transmission that provide a formalism to study the evolution of honest signaling and deception between a sender and a receiver. In many practical scenarios, lies often affect groups of receivers, which inevitably entangles the payoffs of individuals to the payoffs of other agents in their group, and this makes the formalism of pairwise sender-receiver games inapt for where it might be useful the most. We therefore introduce group interactions among receivers and study how their interconnectedness in higher-order social networks affects the evolution of lying. We observe a number of counterintuitive results that are rooted in the complexity of the underlying evolutionary dynamics, which has thus far remained hidden in the realm of pairwise interactions. We find conditions for honesty to persist even when there is a temptation to lie, and we observe the prevalence of moral strategy profiles even when lies favor the receiver at a cost to the sender. We confirm the robustness of our results by further performing simulations on hypergraphs created from real-world data using the SocioPatterns database. Altogether, our results provide persuasive evidence that moral behavior may evolve on higher-order social networks, at least as long as individuals interact in groups that are small compared to the size of the network