Sample-Efficient Model-Free Reinforcement Learning with Off-Policy Critics

Value-based reinforcement-learning algorithms provide state-of-the-art results in model-free discrete-action settings, and tend to outperform actor-critic algorithms. We argue that actor-critic algorithms are limited by their need for an on-policy critic. We propose Bootstrapped Dual Policy Iteration (BDPI), a novel model-free reinforcement-learning algorithm for continuous states and discrete actions, with an actor and several off-policy critics. Off-policy critics are compatible with experience replay, ensuring high sample-efficiency, without the need for off-policy corrections. The actor, by slowly imitating the average greedy policy of the critics, leads to high-quality and state-specific exploration, which we compare to Thompson sampling. Because the actor and critics are fully decoupled, BDPI is remarkably stable, and unusually robust to its hyper-parameters. BDPI is significantly more sample-efficient than Bootstrapped DQN, PPO, and ACKTR, on discrete, continuous and pixel-based tasks. Source code: https://github.com/vub-ai-lab/bdpi.Comment: Accepted at the European Conference on Machine Learning 2019 (ECML

Evolutionary testing supported by slicing and transformation

Evolutionary testing is a search based approach to the automated generation of systematic test data, in which the search is guided by the test data adequacy criterion. Two problems for evolutionary testing are the large size of the search space and structural impediments in the implementation of the program which inhibit the formulation of a suitable fitness function to guide the search. In this paper we claim that slicing can be used to narrow the search space and transformation can be applied to the problem of structural impediments. The paper presents examples of how these two techniques have been successfully employed to make evolutionary testing both more efficient and more effective

A saposin-lipoprotein nanoparticle system for membrane proteins.

A limiting factor in membrane protein research is the ability to solubilize and stabilize such proteins. Detergents are used most often for solubilizing membrane proteins, but they are associated with protein instability and poor compatibility with structural and biophysical studies. Here we present a saposin-lipoprotein nanoparticle system, Salipro, which allows for the reconstitution of membrane proteins in a lipid environment that is stabilized by a scaffold of saposin proteins. We demonstrate the applicability of the method on two purified membrane protein complexes as well as by the direct solubilization and nanoparticle incorporation of a viral membrane protein complex from the virus membrane. Our approach facilitated high-resolution structural studies of the bacterial peptide transporter PeptTSo2 by single-particle cryo-electron microscopy (cryo-EM) and allowed us to stabilize the HIV envelope glycoprotein in a functional state

From Rotating Atomic Rings to Quantum Hall States

Considerable efforts are currently devoted to the preparation of ultracold neutral atoms in the emblematic strongly correlated quantum Hall regime. The routes followed so far essentially rely on thermodynamics, i.e. imposing the proper Hamiltonian and cooling the system towards its ground state. In rapidly rotating 2D harmonic traps the role of the transverse magnetic field is played by the angular velocity. For particle numbers significantly larger than unity, the required angular momentum is very large and it can be obtained only for spinning frequencies extremely near to the deconfinement limit; consequently, the required control on experimental parameters turns out to be far too stringent. Here we propose to follow instead a dynamic path starting from the gas confined in a rotating ring. The large moment of inertia of the fluid facilitates the access to states with a large angular momentum, corresponding to a giant vortex. The initial ring-shaped trapping potential is then adiabatically transformed into a harmonic confinement, which brings the interacting atomic gas in the desired quantum Hall regime. We provide clear numerical evidence that for a relatively broad range of initial angular frequencies, the giant vortex state is adiabatically connected to the bosonic $\nu=1/2$ Laughlin state, and we discuss the scaling to many particles.Comment: 9 pages, 5 figure

Generalized Mittag-Leffler Distributions and Processes for Applications in Astrophysics and Time Series Modeling

Geometric generalized Mittag-Leffler distributions having the Laplace transform $\frac{1}{1+\beta\log(1+t^\alpha)},00$ is introduced and its properties are discussed. Autoregressive processes with Mittag-Leffler and geometric generalized Mittag-Leffler marginal distributions are developed. Haubold and Mathai (2000) derived a closed form representation of the fractional kinetic equation and thermonuclear function in terms of Mittag-Leffler function. Saxena et al (2002, 2004a,b) extended the result and derived the solutions of a number of fractional kinetic equations in terms of generalized Mittag-Leffler functions. These results are useful in explaining various fundamental laws of physics. Here we develop first-order autoregressive time series models and the properties are explored. The results have applications in various areas like astrophysics, space sciences, meteorology, financial modeling and reliability modeling.Comment: 12 pages, LaTe

VEZF1 elements mediate protection from DNA methylation

There is growing consensus that genome organization and long-range gene regulation involves partitioning of the genome into domains of distinct epigenetic chromatin states. Chromatin insulator or barrier elements are key components of these processes as they can establish boundaries between chromatin states. The ability of elements such as the paradigm β-globin HS4 insulator to block the range of enhancers or the spread of repressive histone modifications is well established. Here we have addressed the hypothesis that a barrier element in vertebrates should be capable of defending a gene from silencing by DNA methylation. Using an established stable reporter gene system, we find that HS4 acts specifically to protect a gene promoter from de novo DNA methylation. Notably, protection from methylation can occur in the absence of histone acetylation or transcription. There is a division of labor at HS4; the sequences that mediate protection from methylation are separable from those that mediate CTCF-dependent enhancer blocking and USF-dependent histone modification recruitment. The zinc finger protein VEZF1 was purified as the factor that specifically interacts with the methylation protection elements. VEZF1 is a candidate CpG island protection factor as the G-rich sequences bound by VEZF1 are frequently found at CpG island promoters. Indeed, we show that VEZF1 elements are sufficient to mediate demethylation and protection of the APRT CpG island promoter from DNA methylation. We propose that many barrier elements in vertebrates will prevent DNA methylation in addition to blocking the propagation of repressive histone modifications, as either process is sufficient to direct the establishment of an epigenetically stable silent chromatin stat

Solving ill-posed bilevel programs

This paper deals with ill-posed bilevel programs, i.e., problems admitting multiple lower-level solutions for some upper-level parameters. Many publications have been devoted to the standard optimistic case of this problem, where the difficulty is essentially moved from the objective function to the feasible set. This new problem is simpler but there is no guaranty to obtain local optimal solutions for the original optimistic problem by this process. Considering the intrinsic non-convexity of bilevel programs, computing local optimal solutions is the best one can hope to get in most cases. To achieve this goal, we start by establishing an equivalence between the original optimistic problem an a certain set-valued optimization problem. Next, we develop optimality conditions for the latter problem and show that they generalize all the results currently known in the literature on optimistic bilevel optimization. Our approach is then extended to multiobjective bilevel optimization, and completely new results are derived for problems with vector-valued upper- and lower-level objective functions. Numerical implementations of the results of this paper are provided on some examples, in order to demonstrate how the original optimistic problem can be solved in practice, by means of a special set-valued optimization problem

Constructing Dirac linear fermions in terms of non-linear Heisenberg spinors

We show that the massive (or massless) neutrinos can be described as special states of Heisenberg nonlinear spinors. As a by-product of this decomposition a particularly attractive consequence appears: the possibility of relating the existence of only three species of mass-less neutrinos to such internal non-linear structure. At the same time it allows the possibility that neutrino oscillation can occurs even for massless neutrinos

Facile Synthesis of High Quality Graphene Nanoribbons

Graphene nanoribbons have attracted attention for their novel electronic and spin transport properties1-6, and because nanoribbons less than 10 nm wide have a band gap that can be used to make field effect transistors. However, producing nanoribbons of very high quality, or in high volumes, remains a challenge. Here, we show that pristine few-layer nanoribbons can be produced by unzipping mildly gas-phase oxidized multiwalled carbon nanotube using mechanical sonication in an organic solvent. The nanoribbons exhibit very high quality, with smooth edges (as seen by high-resolution transmission electron microscopy), low ratios of disorder to graphitic Raman bands, and the highest electrical conductance and mobility reported to date (up to 5e2/h and 1500 cm2/Vs for ribbons 10-20 nm in width). Further, at low temperature, the nanoribbons exhibit phase coherent transport and Fabry-Perot interference, suggesting minimal defects and edge roughness. The yield of nanoribbons was ~2% of the starting raw nanotube soot material, which was significantly higher than previous methods capable of producing high quality narrow nanoribbons1. The relatively high yield synthesis of pristine graphene nanoribbons will make these materials easily accessible for a wide range of fundamental and practical applications.Comment: Nature Nanotechnology in pres

An extracellular steric seeding mechanism for Eph-ephrin signaling platform assembly

Erythropoetin-producing hepatoma (Eph) receptors are cell-surface protein tyrosine kinases mediating cell-cell communication. Upon activation, they form signaling clusters. We report crystal structures of the full ectodomain of human EphA2 (eEphA2) both alone and in complex with the receptor-binding domain of the ligand ephrinA5 (ephrinA5 RBD). Unliganded eEphA2 forms linear arrays of staggered parallel receptors involving two patches of residues conserved across A-class Ephs. eEphA2-ephrinA5 RBD forms a more elaborate assembly, whose interfaces include the same conserved regions on eEphA2, but rearranged to accommodate ephrinA5 RBD. Cell-surface expression of mutant EphA2s showed that these interfaces are critical for localization at cell-cell contacts and activation-dependent degradation. Our results suggest a 'nucleation' mechanism whereby a limited number of ligand-receptor interactions 'seed' an arrangement of receptors which can propagate into extended signaling arrays