Optimistic Agents are Asymptotically Optimal

We use optimism to introduce generic asymptotically optimal reinforcement learning agents. They achieve, with an arbitrary finite or compact class of environments, asymptotically optimal behavior. Furthermore, in the finite deterministic case we provide finite error bounds.Comment: 13 LaTeX page

On the Computability of Solomonoff Induction and Knowledge-Seeking

Solomonoff induction is held as a gold standard for learning, but it is known to be incomputable. We quantify its incomputability by placing various flavors of Solomonoff's prior M in the arithmetical hierarchy. We also derive computability bounds for knowledge-seeking agents, and give a limit-computable weakly asymptotically optimal reinforcement learning agent.Comment: ALT 201

Extreme State Aggregation Beyond MDPs

We consider a Reinforcement Learning setup where an agent interacts with an environment in observation-reward-action cycles without any (esp.\ MDP) assumptions on the environment. State aggregation and more generally feature reinforcement learning is concerned with mapping histories/raw-states to reduced/aggregated states. The idea behind both is that the resulting reduced process (approximately) forms a small stationary finite-state MDP, which can then be efficiently solved or learnt. We considerably generalize existing aggregation results by showing that even if the reduced process is not an MDP, the (q-)value functions and (optimal) policies of an associated MDP with same state-space size solve the original problem, as long as the solution can approximately be represented as a function of the reduced states. This implies an upper bound on the required state space size that holds uniformly for all RL problems. It may also explain why RL algorithms designed for MDPs sometimes perform well beyond MDPs.Comment: 28 LaTeX pages. 8 Theorem

Universal knowledge-seeking agents for stochastic environments

We define an optimal Bayesian knowledge-seeking agent, KL-KSA, designed for countable hypothesis classes of stochastic environments and whose goal is to gather as much information about the unknown world as possible. Although this agent works for arbitrary countable classes and priors, we focus on the especially interesting case where all stochastic computable environments are considered and the prior is based on Solomonoff’s universal prior. Among other properties, we show that KL-KSA learns the true environment in the sense that it learns to predict the consequences of actions it does not take. We show that it does not consider noise to be information and avoids taking actions leading to inescapable traps. We also present a variety of toy experiments demonstrating that KL-KSA behaves according to expectation

Bayesian reinforcement learning with exploration

We consider a general reinforcement learning problem and show that carefully combining the Bayesian optimal policy and an exploring policy leads to minimax sample-complexity bounds in a very general class of (history-based) environments. We also prove lower bounds and show that the new algorithm displays adaptive behaviour when the environment is easier than worst-case

Irus and his jovial crew : representations of beggars in Vincent Bourne and other eighteenth-century writers of Latin verse

Alastair Fowler has written, with reference to the time of Milton, of ‘Latin's special role in a bilingual culture’, and this was still true in the early eighteenth century. The education of the elite placed great emphasis on the art of writing Latin verse and modern, as well as ancient, writers of Latin continued to be widely read. Collections of Latin verse, by individual writers such as Vincent Bourne (c. 1694–1747) or by groups such as Westminster schoolboys or bachelors of Christ Church, Oxford, could run into multiple editions, and included poems on a wide range of contemporary topics, as well as reworkings of classical themes. This paper examines a number of eighteenth-century Latin poems dealing with beggars, several of which are here translated for the first time. Particular attention is paid to the way in which the Latin poems recycled well-worn tropes about beggary which were often at variance with the experience of real-life beggars, and to how the specificities of Latin verse might heighten negative representations of beggars in a genre which, as a manifestation of elite culture, appealed to the very class which was politically and legally responsible for controlling them

Sequential Extensions of Causal and Evidential Decision Theory

Moving beyond the dualistic view in AI where agent and environment are separated incurs new challenges for decision making, as calculation of expected utility is no longer straightforward. The non-dualistic decision theory literature is split between causal decision theory and evidential decision theory. We extend these decision algorithms to the sequential setting where the agent alternates between taking actions and observing their consequences. We find that evidential decision theory has two natural extensions while causal decision theory only has one.Comment: ADT 201

Adolescents' views of food and eating: Identifying barriers to healthy eating

This is a postprint version of the article. The official published version can be accessed from the link below - © 2006 The Association for Professionals in Services for Adolescents Published by Elsevier Ltd.Contemporary Western society has encouraged an obesogenic culture of eating amongst youth. Multiple factors may influence an adolescent's susceptibility to this eating culture, and thus act as a barrier to healthy eating. Given the increasing prevalence of obesity amongst adolescents, the need to reduce these barriers has become a necessity. Twelve focus group discussions of single-sex groups of boys or girls ranging from early to-mid adolescence (N = 73) were employed to identify key perceptions of, and influences upon, healthy eating behaviour. Thematic analysis identified four key factors as barriers to healthy eating. These factors were: physical and psychological reinforcement of eating behaviour; perceptions of food and eating behaviour; perceptions of contradictory food-related social pressures; Q perceptions of the concept of healthy eating itself. Overall, healthy eating as a goal in its own right is notably absent from the data and would appear to be elided by competing pressures to eat unhealthily and to lose weight. This insight should inform the development of future food-related communications to adolescents. (c) 2006 The Association for Professionals in Services for Adolescents.Funding from Safefood: the food safety promotion board is acknowledged

The diverse nature of island isolation and its effect on land bridge insular faunas

Aim: Isolation is a key factor in island biology. It is usually defined as the distance to the geographically nearest mainland, but many other definitions exist. We explored how testing different isolation indices affects the inference of impacts of isolation on faunal characteristics. We focused on land bridge islands and compared the relationships of many spatial and temporal (i.e., through time) isolation indices with community‐, population‐ and individual‐level characteristics (species richness, population density and body size, respectively). Location: Aegean Sea islands, Greece. Time period: Current. Taxon: Many animal taxa. Methods: We estimated 21 isolation indices for 205 islands and recorded species richness data for 15 taxa (invertebrates and vertebrates). We obtained body size data for seven lizard species and population density data for three. We explored how well indices predict each characteristic, in each taxon, by conducting a series of ordinary least squares regressions (controlling for island area when needed) and a meta‐analysis. Results: Isolation was significantly (and negatively) associated with species richness in 10 of 15 taxa. It was significantly (and positively) associated with body size in only one of seven species and was not associated with population density. The effect of isolation on species richness was much weaker than that of island area, regardless of the index tested. Spatial indices generally out‐performed temporal indices, and indices directly related to the mainland out‐performed those related mainly to neighbouring islands. No index was universally superior to others, including the distance to the geographically nearest mainland. Main conclusions: The choice of index can alter our perception of the impacts of isolation on biological patterns. The nearly automatic, ubiquitous use of distance to the geographically nearest mainland misrepresents the complexity of the effects of isolation. We recommend the simultaneous testing of several indices that represent different aspects of isolation, in order to produce more constructive and thorough investigations and avoid imprecise inference